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1. INTRODUCTION 



This methodology report provides technical information about the development, design, and 
conduct of the fifth-grade 1 data collection of the Early Childhood Longitudinal Study, Kindergarten Class 
of 1998-99 (ECLS-K). It begins with an overview of the ECLS-K study. Subsequent chapters provide 
information on the development of the instruments, sample design, data collection methods, data 
preparation and editing, response rates, and weighting and variance estimation. 

The ECLS-K focuses on children’s early school experiences, beginning with kindergarten. It 
is a multisource, multimethod study that includes interviews with parents; the collection of data from 
principals, teachers, and student record abstracts; and direct child assessments. The ECLS-K was 
developed under the sponsorship of the U.S. Department of Education, National Center for Education 
Statistics (NCES) in the Institute of Education Sciences. Westat is conducting this study with assistance 
provided by Educational Testing Service (ETS) of Princeton, New Jersey. The Survey Research Center 
and the School of Education at the University of Michigan assisted Westat in conducting the base year 
and first-grade studies. 

The ECLS-K follows a nationally representative cohort of children from kindergarten into 
high school. The base year data were collected in the fall and spring of the 1998-99 school year when the 
sampled children were in kindergarten. A total of 21,260 kindergartners throughout the nation 
participated. 



Two more waves of data were collected in the fall and spring of the 1999-2000 school year 
when most, but not all, of the base year children were in first grade. 2 The fall-first grade data collection 
was limited to a 30 percent subsample of schools. Approximately 27 percent of the base year students 
who were eligible to participate in year 2 attended the 30 percent subsample of schools (see exhibit 1-1). 
The fall-first grade data collection was a design enhancement to enable researchers to measure the extent 
of summer learning loss and the factors that contributed to such loss and to better disentangle school and 
home effects on children’s learning. The spring-first grade data collection, including the full sample, was 



1 The term “fifth grade” is used throughout this document to refer to the data collection that took place in the 2003-2004 school year, at which 
time most of the sampled children — but not all of them — were in fifth grade. 

2 Though the majority of base year children were in first grade during the 1999-2000 school year, about 5 percent of the sampled children were 
retained in kindergarten and a handful of others were in second grade during the 1 999-2000 school year. 
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Exhibit 1-1. ECLS-K waves of data collection: School years 1998-99, 1999-2000, 2001-02, and 
2003-04 



Data collection 


Date of collection 


Sample 


F all-kindergarten 


Fall 1998 


Full sample 


Spring-kindergarten 


Spring 1999 


Full sample 


Fall-first grade 


Fall 1999 


30 percent subsample 1 


Spring-first grade 


Spring 2000 


Full sample 


Spring-third grade 


Spring 2002 


Full sample 


Spring-fifth grade 


Spring 2004 


Full sample 



1 Fall data collection consisted of a 30 percent sample of schools containing approximately 27 percent of the base year students eligible to 
participate in year 2. 



NOTE: See section 1.3 for a description of the study components. More information is provided in the Combined User’s Manual for the ECLS-K 
Fifth-Grade Data Files and Electronic Codebooks (NCES 2006-032) (Tourangeau et al. forthcoming). 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99, spring 2004. 



part of the original study design and can be used to measure annual school progress and to describe the 
first grade learning environment of children in the study. All children assessed during the base year were 
eligible to be assessed in the spring-first grade data collection regardless of whether they repeated 
kindergarten, were promoted to first grade, or were promoted to second grade. In addition, children who 
were not in kindergarten in the United States during the 1998-99 school year and, therefore, did not have 
a chance to be selected to participate in the base year of the ECLS-K were added to the spring-first grade 
sample. 3 Such children included immigrants to the United States who arrived after fall 1998 sampling, 
children living abroad during the 1998-99 school year, children who were in first grade in 1998-99 and 
repeated it in 1999-2000, and children who did not attend kindergarten. Their addition allows researchers 
to make estimates for all first-graders in the United States rather than just for those who attended 
kindergarten in the United States in the previous year. 

A fifth wave of data was collected in the spring of the 2001-02 school year when most, but 
not all, of the sampled children were in third grade. Approximately 89 percent of the children interviewed 
were in third grade during the 2001-02 school year, 9 percent were in second grade, and less than 1 
percent were in fourth grade or higher. In addition to the school, teacher, parent, and child assessment 
data collection components, children were asked to complete a short self-description questionnaire, which 
asked them how they thought and felt about themselves, both socially and academically. The spring-third 



3 The addition of these children is referred to as “freshening” the sample. See chapter 3 for more detail on the freshening process. 
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grade data collection can be used to measure school progress and to describe the third-grade learning 
environment of children in the study. 

A sixth wave of data was collected in the spring of the 2003-04 school year when most, but 
not all, of the sampled children were in fifth grade. 4 In addition to the school, teacher, parent, and child 
assessment data collection components, children were asked to complete a short self-description 
questionnaire, which asked them how they thought and felt about themselves, both socially and 
academically. They were also asked about their food consumption at school and in the week prior to the 
interview. The spring-fifth grade data collection can be used to measure school progress and to describe 
the fifth-grade learning environment of children in the study. 

The sample of children in the fifth-grade round of data collection of the ECLS-K represents 
the cohort of children who were in kindergarten in 1998-99 or in first grade in 1999-2000. Since the 
sample was not freshened after the first-grade year with third- or fifth-graders who did not have a chance 
to be sampled in kindergarten or first grade (as was done in first grade), estimates from the ECLS-K third- 
and fifth-grade data are representative of the population cohort rather than all third-graders in 2001-02 or 
all fifth-graders in 2003-04. The estimated number of third-graders from the third-grade ECLS-K data 
collection is approximately 86 percent of all third-graders. From the fifth-grade ECLS-K data collection, 
the estimated number of fifth-graders is approximately 83 percent of all fifth-graders. While the vast 
majority of children in third grade in the 2001-02 school year and in fifth grade in the 2003-04 school 
year are members of the cohort, third-graders who repeated second or third grade, fifth-graders who 
repeated third or fourth grade, and recent immigrants are not covered. Data were collected from teachers 
and schools to provide important contextual information about the school environment for the sampled 
children. The teachers and schools are not representative of fifth-grade teachers and schools in the country 
in 2003-04. For this reason, the only weights produced from the study are for making statements about 
children, including statements about the teachers and schools of those children. 

The ECLS-K has several major objectives and numerous potential applications. The 
ECLS-K combines (1) a study of achievement in the elementary years; (2) an assessment of the 
developmental status of children in the United States at the start of their formal schooling and at key 
points during the elementary school years; (3) cross-sectional studies of the nature and quality of 
kindergarten programs in the United States; and (4) a study of the relationship of family, preschool, and 



4 Approximately 90 percent of the children interviewed were in fifth grade during the 2003-04 school year, 9 percent were in fourth grade, and 
less than 1 percent were in third or some other grade. 
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school experiences to children’s developmental status at school entry and their progress from 
kindergarten, through elementary school, and into high school. 



The ECLS-K is part of a longitudinal studies program comprising two cohorts — a 
kindergarten cohort and a birth cohort. The birth cohort (ECLS-B) is following a national sample of 
children bom in the year 2001 from birth to kindergarten. The ECLS-B focuses on the characteristics of 
children and their families that influence children’s first experiences with the demands of formal school, 
as well as children’s early health care and in- and out-of-home experiences. Together these cohorts will 
provide the depth and breadth of data required to more fully describe and understand children’s health and 
early learning, development, and education experiences. 

The ECLS-K has both descriptive and analytic purposes. It provides descriptive data on 
children’s status at school entry, their transition into school, and their progress into high school. The 
ECLS-K also provides a rich data set that enables researchers to analyze how a wide range of family, 
school, community, and individual variables affect children’s early success in school; explore school 
readiness and the relationship between the kindergarten experience and later elementary school 
performance; and record children’s cognitive and academic growth as they move through secondary 
school. 



1.1 Background 

Efforts to expand and improve early education will benefit from insights gained through 
analyses drawn from the large scale, nationally representative ECLS-K data and the study’s longitudinal 
design. The ECLS-K database contains information about the types of school programs in which children 
participated, the services they received, and repeated measures of the children’s cognitive skills and 
knowledge. The ECLS-K database also contains measures of children’s physical health and growth, social 
development, and emotional well-being, along with information on family background and the 
educational quality of their home environments. 

As a study of early achievement, the ECLS-K allows researchers to examine how children’s 
progress is associated with such factors as placement in high or low ability groups, receipt of special 
services or remedial instruction, grade retention, and frequent changes in schools attended because of 
family moves. Data on these early school experiences are collected as they occur, with the exception of 
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their experiences before kindergarten, which were collected retrospectively. This produces a more 
accurate measurement of these antecedent factors and enables inferences to be made about their 
relationship to later academic progress. The longitudinal nature of the study enables researchers to study 
children’s cognitive, social, and emotional growth and to relate trajectories of change to variations in 
children’s experiences in kindergarten and the early grades to later grades. 

The spring-fifth grade data collection can be used to describe the diversity of children in the 
study and the classrooms and schools they attend. It can also be used to study children’s academic gains 
in the years following kindergarten and first grade. The ECLS-K sample includes substantial numbers of 
children from various minority groups. Thus, the ECLS-K data present many possibilities for studying 
cultural and ethnic differences in the educational preferences and literacy practices of families, the 
developmental patterns and learning styles of children, and the educational resources and opportunities 
that different groups are afforded in the United States. 



1.2 Conceptual Model 

The design of the ECLS-K has been guided by a framework of children’s development and 
schooling that emphasizes the interrelationships between the child and family, the child and school, the 
family and school, and the family, school, and community. The ECLS-K recognizes the importance of 
factors that represent the child’s health status and socioemotional and intellectual development and 
incorporates factors from the child’s family, community, and school-classroom environments. The 
ECLS-K conceptual model is depicted in exhibit 1-2. The study has paid particular attention to the role 
that parents and families play in helping children adjust to formal school and in supporting their education 
through the elementary grades. It has also gathered information on how schools prepare for and respond 
to the diverse backgrounds and experiences of the children and families they serve. 



1.3 Study Components 

The emphasis placed on measuring children’s environments and development broadly has 
critical implications for the design of the ECLS-K. The design of the study includes the collection of data 
from the child, the child’s parents/guardians, teachers, and schools. 
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Exhibit 1-2. ECLS-K conceptual model: School years 1998-2004 




SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998 - 99 . 



■ Children participate in various activities to measure the extent to which they exhibit 
those abilities and skills deemed important for success in school. They are asked to 
participate in activities designed to measure important cognitive (i.e., literacy, 
quantitative, and science) skills and noncognitive (fine and gross motor coordination 
[in kindergarten] and socioemotional) skills and knowledge. Most measures of a 
child’s cognitive skills are obtained through an untimed one-on-one assessment of the 
child. Beginning with the third-grade data collection, children report on their own 
perceptions of their abilities and achievement as well as their interest in and 
enjoyment of reading, math, and other school subjects. Children are assessed in each 
round of data collection. Children’s height and weight are also measured in each 
round of data collection. 

■ Parents/guardians are an important source of information about the families of the 
children selected for the study and about themselves. Parents provide information 
about children’s development at school entry and their experiences both with family 
members and others. Information is collected from parents each time children are 
assessed using computer-assisted interviews (CAIs). Information is collected from 
parents/guardians in each round of data collection. 

■ Teachers, like parents, represent a valuable source of information on themselves, the 
children in their classrooms, and the children’s learning environment (i.e., the 
classroom). Teachers are not only asked to provide information about their own 
backgrounds, teaching practices, and experience, they are also called on to provide 
information on the classroom setting for the sampled children they teach and to 
evaluate each sampled child on a number of critical cognitive and noncognitive 
dimensions. Special education teachers and service providers of sampled children with 
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disabilities are also asked to provide information on the nature and types of services 
provided to the child. With the exception of the fall-first grade data collection, 
teachers complete self-administered questionnaires each time children are assessed. 

■ School administrators, or their designees, are asked to provide information on the 
physical, organizational, and fiscal characteristics of their schools, and on the schools’ 
learning environment and programs. Special attention is paid to the instructional 
philosophy of the school and its expectations for students. School administrators or 
their designees, are also asked to provide basic information about school 5 grade level, 
school type (public or private), length of school year, and attendance recordkeeping 
practices. Information is collected from school administrators via self-administered 
questionnaires during each spring data collection. 

■ School office staff are asked to complete a student records abstract form and provide 
basic information about the school. The student records abstract form includes 
questions about an individual child’s enrollment and attendance at the school, transfer 
to another school (if applicable), and verifies whether the child has an Individualized 
Education Program (IEP) on record. A student records abstract form is completed for 
each child in the study during each spring data collection. 

Exhibit 1 -3 summarizes the instruments that were used in each of the data collection periods 
from kindergarten through spring-fifth grade. Exhibit 1-4 provides additional detail about the direct child 
assessments conducted during each of the data collection periods. Separate psychometric reports have 
been prepared to describe the design and development of the kindergarten through first-grade and third- 
grade assessment batteries. For detailed information about the child assessments, including their 
psychometric properties, see the ECLS-K Psychometric Report for Kindergarten Through First Grade, 
NCES 2002-05 (Rock and Pollack 2002); the ECLS-K Psychometric Report for the Third Grade (NCES 

2005- 062) (Pollack et al. 2005); and the ECLS-K Psychometric Report for the Fifth Grade (NCES 

2006- 036) (Pollack et al. 2005). 



1.4 Contents of Report 

This report provides detailed technical information about the development, design, and 
conduct of the fifth-grade data collection. Chapter 2 provides an overview of processes used to develop 
the computer-assisted (CAI) and hard-copy survey instruments. Chapter 3 describes the sample design 
and implementation. Chapter 4 describes the data collection methods, including information about the 
training of field staff and quality control procedures. Chapter 5 details the preparation and editing of the 



5 These items were collected in a separate school fact sheet in 3 rd grade, but were reintegrated into the school administrator questionnaire in the 
5 th grade data collection. Prior to the 3 rd grade data collection, the questions had been part of the school administrator questionnaire. 
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data as it is receipted from the field. Chapter 6 provides information on unit and item response rates. 
Chapter 7 discusses weighting and variance information. 

Because both this report and the ECLS-K Psychometric Report for the Fifth Grade focus on 
the fifth-grade data collection, minimal information is provided about the base year, first-grade, or third- 
grade data. Users who wish to learn more about these data collections should refer to the ECLS-K Base 
Year Public-Use Data Files and Electronic Codebook: User’s Manual (NCES 2001-029r) (Tourangeau, 
Burke, et al. 2004); the User ’s Manual for the ECLS-K First Grade Public-Use Data Files and Electronic 
Codebook (NCES 2002-135) (Tourangeau et al. 2002); or the User’s Manual for the ECLS-K Third 
Grade Public-Use Data File and Electronic Code Book (NCES 2004-001) (Tourangeau, Brick, et al. 
2004) Additional information about the ECLS program can be found on the World Wide Web at 
http://nces.ed.gov/ecls . 
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Exhibit 1-3. Instruments used in the ECLS-K, by round of data collection: School years 1998-99, 
1999-2000, 2001-02, and 2003-04 





1998-99 


1999- 


K) 

o 

o 

o 


2001-02 


2003-04 




school year 


school year 


school year 


school year 




Fall- 


Spring- 


Fall- 


Spring- 


Spring- 


Spring- 


Instruments 


kindergarten 


kindergarten 


first grade 1 


first grade 2 


third grade 


fifth grade 



Parent interview 


X X 


X 


X 


X 


X 


Child assessments 


X X 


X 


X 


X 


X 


Teacher questionnaire part A 


X X 


X 


X 2 


X 




Teacher questionnaire part B 


X X 


X 


X 2 


X 




Teacher questionnaire part C 


X X 


X 


X 2 


X 




Teacher questionnaire (teacher level) 










X 3 


Reading teacher questionnaire 










X 


Math teacher questionnaire 










X 


Science teacher questionnaire 










X 


Special education teacher 












questionnaire part A 


X 




X 


X 


X 


Special education teacher 












questionnaire part B 


X 




X 


X 


X 


Adaptive Behavior Scale 


X 




X 






Self-description questionnaire 








X 


X 


Food consumption questionnaire 










X 


School administrator questionnaire 


X 




X 4 


X 


X 5 


Student record abstract 


X 




X 


X 


X 


School fact sheet 








X 6 




School facilities checklist 


X 




X 


X 


X 


Salary and benefits questionnaire 7 


X 










Head Start verification 8 


X 











X Round that included the instrument. 

1 The fall-first grade data collection consisted of a 30 percent subsample of the study schools. See the User’s Manual for the ECLS-K First Grade 
Public-Use Data Files and Electronic Code Book (NCES 2002-135) (Tourangeau, Burke, et al. 2002) for information about the purposes and 
methods of the fall-first grade data collection. 

2 In spring-first grade, there were two sets of teacher questionnaires — one for the teachers of children who had made the transition to the first 
grade or any higher elementary school grade, and the second for teachers of children who were repeating or attending the second year of 
kindergarten. 

3 In spring-fifth grade, teacher questionnaires part A, B, and C were replaced by a teacher-level questionnaire and questionnaires for reading, 
math, and science teachers. 

4 In spring-first grade, there were two different school administrator questionnaires — one for school administrators in schools new to the study 
and one for school administrators in schools that participated in the base year data collection. 

5 In spring-fifth grade, questions from the school fact sheet used in spring-third grade were included in the school administrator questionnaire. 

6 The items in the school fact sheet were included in the school administrator questionnaire in kindergarten and in first grade. These items were 
reintegrated into the school administrator questionnaire in the 5 th grade data collection. 

7 The salary and benefits questionnaire collected information on the base salary, merit pay, and health benefit pay of teachers and principals. It 
was completed by the school or district business administrator or by a private school administrator or headmaster. 

8 The Head Start Verification Study confirmed parent and school reports of children’s Head Start participation by matching information on the 
name and location of the Head Start facilities the children were reported to have attended against a database of Head Start centers. For each 
match, the center was contacted to confirm that the child had attended the center in the year before kindergarten. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), fall 1998, spring 1999, fall 1999, spring 2000, spring 2002, and spring 2004. 
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Exhibit 1-4. Direct child assessments, by domain and round of data collection: School years 1998-99, 
1999-2000, 2001-02, and 2003-04 



Direct child assessment 


1998-99 
school year 


1999-2000 
school year 


2001-02 
school year 


2003-04 
school year 


Fall- 

kindergarten 


Spring- 

kindergarten 


Fall- 
first grade 


Spring- 
first grade 


Spring- 
third grade 


Spring- 
fifth grade 


Language screener (Oral Language 














Development ScalefOLDS]) 1 


X 


X 


X 


X 






Food consumption 














questionnaire (FCQ) 












X 


Reading (language and literacy) 


X 


X 


X 


X 


X 


X 


Mathematical thinking 


X 


X 


X 


X 


X 


X 


Socioemotional development 










X 


X 


General knowledge (science and 














social studies) 


X 


X 


X 


X 






Science 2 










X 


X 


Psychomotor 


X 












Height and weight 


X 


X 


X 


X 


X 


X 



X Round that included the instrument. 

1 OLDS (Oral Language Development Scale) was given to language-minority students new to the study in the spring, or who did not pass the cut 
score in the English version during the previous OLDS administration. The screener determined if the children understood English well enough to 
receive the direct child assessments in English. For further information on the language screener, please refer to the ECLS-K Base Year Public- 
Use Data Files and Electronic Code Book: User’s Manual (NCES 2001-029r) (Tourangeau; Burke, et al. 2004). The screener was not used in 
third or fifth grade because the vast majority of children passed it by spring-first grade. 

2 In spring-third grade, the general knowledge assessment was replaced with a science assessment. Children received a science assessment in 
third and fifth grade that measured their understanding of science concepts and scientific investigation skills. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), fall 1998, spring 1999, fall 1999, spring 2000, spring 2002, and spring 2004. 
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2. DEVELOPMENT OF SURVEY INSTRUMENTS 



The Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K) fifth- 
grade survey collected data on the achievement and home and school experiences of children who had 
attended kindergarten in 1998-99 to provide information on the children’s progress in the middle 
elementary grades. In the design phase of the ECLS-K kindergarten, first-grade, third-grade, and fifth- 
grade waves of data collection, policymakers, teachers, and researchers were consulted, and relevant 
literature was reviewed to ascertain the specific areas within each of the topical components for which 
national data were needed. Information gathered from these activities guided the formulation of research 
questions deemed most important for the ECLS-K to address. Extant surveys were reviewed to identify 
surveys that had been fielded to answer similar questions. 

The ECLS-K data collection instruments were similar in content and form in all six waves of 
the study. The ECLS-K employed two modes of data collection, computer-assisted and self-administered 
hard-copy instruments. This chapter describes the development of the computer-assisted and hard-copy 
instruments for the fifth-grade data collection. The procedures for developing the child assessment battery 
and indirect rating forms are described in a separate psychometric report. More information on the 
assessment battery and indirect rating forms is found in ECLS-K Psychometric Report for Kindergarten 
Through First Grade (NCES 2002-05) (Rock and Pollack 2002) and in ECLS-K Psychometric Report for 
the Third Grade (NCES 2005-062) (Pollack et al. 2005). 

In spring-fifth grade, several modifications were made to the instruments. Many of the 
changes were based on advice given by the ECLS-K Technical Review Panel (TRP) and Content Review 
Panel (CRP) that was provided for both the spring-third grade and spring-fifth grade data collections 
simultaneously. Modifications were made to the fifth-grade parent interview to reduce its length and add 
other items of interest. A timing study was conducted to assess the effect of these changes. Several 
changes were also made to the child assessment, the teacher questionnaires, and the school administrator 
questionnaire. These are discussed below. 
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2.1 



Review Panels 



Studies with the scope, complexity, and importance of the ECLS-K require consultation with 
a number of individuals and organizations to address the data needs of policymakers and of those 
performing policy studies and educational research. In addition, consultations with practitioners, content 
area experts, and researchers are necessary to ensure that instruments accurately reflected curricular 
standards and practices. The ECLS-K project staff established and sought guidance from the Technical 
Review Panel and the Content Review Panel. 



2.1.1 Technical Review Panel 

The Technical Review Panel was assembled to provide review and comment on such matters 
as the technical design and implementation of the ECLS-K and policy and research topics that are 
appropriate for the ECLS-K third-grade and fifth-grade data collections. The membership of the 
Technical Review Panel represents a broad range of nonfederal and federal experts in elementary 
education, educational, and family research and policy issues. 



Technical Review Panel Members: 

Karl Alexander, Johns Hopkins University 

Richard Duran, University of California at Santa Barbara 

Nancy Karweit, Johns Hopkins University 

Donna Morrison, Georgetown Public Policy Institute 

Jane Stallings, Texas A&M University 

Deborah Stipek, Stanford University 

U.S. Department of Education: Institute of Education Sciences, National Center 

for Education Statistics 

Office of Special Education Programs, and 

Office of English Language Acquisition, Language Enhancement and 
Academic Achievement for Limited English Proficient Students 

U.S. Department of Agriculture: Economic Research Service, 
and Office of Analysis, Nutrition, and Evaluation 

U.S. Department of Health and Human Services: 

Administration on Children, Youth and Families 
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2.1.2 



Content Review Panel 



In addition to the Technical Review Panel, the Content Review Panel (CRP) was established 
to provide expert review of (1) the validity of the content of the child assessments and (2) the consistency 
of the items in the assessment battery with instructional practice. The panel included subject matter 
experts in reading, mathematics, and science as well as school assessment and evaluation administrators. 



Content Review Panel Members 

Dr. Gloria Johnson, Superintendent, West Contra Costa Unified School 
District, Richmond, California. Specialty: Reading. 

Dr. Jean Joyner, North Carolina Department of Public Instruction. Specialty: 
Mathematics. 

Helen Lounsbury, Teacher, Beme-Knowx-Westerlo Elementary School, Berne, 
New York. Specialty: Mathematics. 

Dr. Alba Ortiz, Professor, Department of Special Education, University of 
Texas. Specialty: Reading. 

Kathy DiRanna, Director, K-12 Alliance, California. Specialty: Science. 

Iris Weiss, President, Horizon Research, Inc., Chapel Hill, North Carolina. 
Specialty: Science. 



2.2 Modifications to the Parent Interview 

Exhibit 2- 1 lists the 1 7 sections comprising the spring-fifth grade parent interview. 

Many items from previous rounds of the ECLS-K were not included in spring-fifth grade 
because they had been used previously and to reduce respondent burden. The following constructs in the 
spring-third grade parent interview were not asked in spring-fifth grade: 

■ School practices (PIQ.030a-e); 

■ School climate (PIQ.120a-h); 
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Exhibit 2-1. Sections comprising the spring- fifth grade parent interview: School year 2003-04 



INQ 

PIQ 

FSQ 

HEQ 

CFQ 

CCQ 

DWQ 

NRQ 

COQ 

CHQ 

PPQ 

FDQ 

PEQ 

EMQ 

WPQ 

PAQ 

CMQ 



Introduction 
Parent Involvement 
Family Structure 

Home Environment, Activities, and Cognitive Stimulation 
Critical Family Processes 
Child Care 

Discipline, Warmth, and Emotional Supportiveness 
Non-Resident Parent Questions 

Country of Origin Questions for Non-Resident Biological Parents 

Child Elealth and Well Being 

Parent’s Psychological Well Being and Elealth 

Food Security 

Parent Education 

Parent Employment 

Welfare and Other Public Transfers 

Parent Income and Assets 

Child Mobility and Plans to Move 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, 
Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 



■ Parent perception of how well child is doing in reading/language arts and math 
(PIQ. 090, PIQ. 100); 

■ Specific home learning activities and resources (HEQ. 010, HEQ. 015, HEQ. 017, 
HEQ.021, HEQ. 023, HEQ.030, HEQ.065B, HEQ.094, HEQ.097, HEQ. 100, 
HEQ. 106); 

■ Family breakfast patterns (HEQ. 115, HEQ. 116, HEQ. 117, HEQ. 118); 

■ Transportation and commute to school (HEQ. 124, HEQ. 125, HEQ. 126, HEQ. 127); 

■ Problems around house/neighborhood (HEQ.410); 

■ Social support (CFQ. 020); and 

■ Parental depression (PPQ.100, PPQ.110, PPQ.120, PPQ.130, PPQ.140, PPQ.150, 
PPQ. 160, PPQ. 170, PPQ. 180, PPQ. 190, PPQ.200, PPQ.210, PPQ.230, PPQ.240). 
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Several new construct areas were added for fifth grade. These were as follows: 



■ Discussions with child about school and friends (HEQ.420a and b); 

■ Discussions with child about smoking, sexual activity, drinking alcohol, and using 
other drugs (HEQ.421a-d); 

■ Identification of when a diagnosis of a particular disability was made, if applicable 
(CHQ.076, CHQ.077, CHQ.136, CHQ.137, CHQ.186, CHQ.187, CHQ.226, 
CHQ.227, CHQ.314, CHQ.315, CHQ.346, CHQ.347, CHQ.376, CHQ.377); 

■ Identification of when cochlear implants were implanted (CHQ.251-CHQ.253); 

■ Child’s use of cochlear implants in school (CHQ.254); 

■ Identification of when child’s use of therapy services or program for children with 
disabilities ended (CHQ.536, CHQ.537); 

■ Reason why the child no longer participated in services for children with special needs 
or special education (CHQ.546); 

■ Whether child takes prescription medication for Attention Deficit Disorder (ADD), 
Attention Deficit Hyperactivity Disorder (ADHD), or hyperactivity (CHQ.740); 

■ Medications taken for ADD, ADHD, or hyperactivity (CHQ.750a-g); 

■ Length of time medications have been taken for ADD, ADHD, or hyperactivity 
(CHQ.760); 

■ Receipt of family therapy (CHQ.770); 

■ Reason for family therapy (CHQ.780); 

■ Type of family therapist seen (CHQ.790); and 

■ Number of times family therapist seen (CHQ.800). 



Other questions were added that had been used previously in the ECLS-K for parent figures 
in the household, but in spring-fifth grade were asked about a new group of persons (nonresident 
biological parents). These were: 



■ Country of origin, age moved to U.S., and U.S. citizenship for non-resident biological 
parents (section COQ); 
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In addition, one question was reintroduced from an earlier year of the study: 



■ Whether father or mother figure has a high school diploma or its equivalent 
(PEQ.021); 

Finally, some questions do not include new content but were modified from a previous 

round: 



■ In spring-fifth grade, interviewers were able to record the time set aside every day for 
children to do homework in hours (HEQ.092b) in addition to minutes (HEQ.093a). 

■ Questions about who usually helps the child with his/her reading, language arts, or 
spelling homework or helps with math homework allowed the respondent to volunteer 
who helped (HEQ.095b, HEQ.099). In spring-third grade, respondents were instead 
asked separate questions about each household member and whether he/she helped 
with homework. 



2.3 Timing Study 

As with any study instrument, questionnaire length and respondent burden were issues of 
concern. A timing study was conducted for the draft parent questionnaire. Three Westat staff members 
conducted nine interviews with respondents who had previously volunteered to participate in studies 
being conducted by Westat. No attempt was made to recruit respondents representative of either racial or 
economic groups as the objective was to obtain an estimate of the length of the questionnaire rather than 
to examine how individuals inteipreted the questions. Westat did attempt to select people who would go 
through the various questionnaire paths (e.g., married couples, single parents). All of the respondents 
were parents of fifth-grade children. All interviews were conducted over the telephone using a paper 
version of the questionnaire. Interviewers used stopwatches to time the individual sections and to get an 
overall time for the interview. The interviewers stopped the watches for extended interruptions, such as a 
respondent having to take care of the needs of a family member. In most cases, the respondents were 
asked to answer questions in sections that required knowledge of data collected from an earlier wave of 
the data collection as if they had provided the information in a previous round of the survey. In only two 
interviews were respondents asked to complete such sections, as would be the case with a new 
respondent. 



The revised paper version of the questionnaire took an average of 39 minutes and 17 seconds 
to complete. Table 2-1 summarizes the overall and section timings for each interviewer and presents the 
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average time expended for each section. The initials denote the three interviewers. Each interviewer 
completed three interviews (e.g., interviewer SG completed SGI, SG2, and SG3). One interview was 
done with a parent of twins (NV2). The second twin’s time was included in the overall average time. 



Table 2-1. Interviewer timings for the revised ECLS-K fifth-grade parent interview, by interviewer and 
by section: School year 2002-03 



Section 1 


SGI 


LB1 


NV1 


LB2 


SG2 


NV2 


NV2 

(TWIN) 


LB3 


SG3 


NV3 


Mean 

time 


Total 


0:35:53 


0:41:18 


0:45:10 


0:41:15 


0:34:23 


0:53:39 


0:14:34 


0:35:14 


0:36:33 


0:30:07 


t 


INQ 


0:01:28 


0:00:39 


0:01:01 


0:01:45 


0:01:12 


0:02:05 


0:00:23 


0:01:31 


0:01:28 


0:00:43 


0:01:19 


PIQ 


0:02:50 


0:02:20 


0:03:28 


0:03:38 


0:03:41 


0:03:58 


0:01:34 


0:03:21 


0:03:04 


0:03:02 


0:03:16 


FSQ 


0:01:59 


0:03:10 


0:01:04 


0:03:30 


0:01:07 


0:02:35 


0:00:00 


0:02:12 


0:01:55 


0:01:39 


0:02:08 


HEQ 


0:08:00 


0:14:05 


0:09:02 


0:07:39 


0:09:18 


0:09:32 


0:04:45 


0:06:43 


0:06:16 


0:07:29 


0:08:40 


CFQ 


0:00:12 


0:00:11 


0:00:00 


0:00:10 


0:00:10 


0:00:56 


t 


0:00:12 


0:00:10 


0:00:09 


0:00:14 


CCQ 


0:01:25 


0:01:00 


0:03:29 


0:02:30 


0:01:53 


0:01:30 


0:00:10 


0:01:14 


0:01:30 


0:01:34 


0:01:47 


DWQ 


0:01:09 


0:01:10 


0:02:12 


0:01:02 


0:01:18 


0:01:15 


0:00:34 


0:00:54 


0:00:59 


0:00:35 


0:01:10 


NRQ 


t 


t 


0:03:27 


t 


t 


t 


t 


t 


0:02:31 


t 


0:02:59 


CHQ 


0:10:17 


0:11:10 


0:13:27 


0:11:30 


0:08:16 


0:08:25 


0:06:59 


0:08:58 


0:07:24 


0:07:20 


0:09:39 


PPQ 


0:00:10 


0:00:07 


0:01:00 


0:00:10 


0:00:10 


0:00:14 


t 


0:00:10 


0:00:10 


0:00:07 


0:00:15 


FDQ 


0:01:04 


0:01:00 


0:01:00 


0:00:51 


0:00:48 


0:01:00 


t 


0:00:54 


0:02:40 


0:00:45 


0:01:07 


PEQ 


0:00:14 


0:00:49 


0:01:00 


0:01:20 


0:00:15 


0:01:00 


t 


0:00:43 


0:00:15 


0:00:52 


0:00:43 


EMQ 


0:01:24 


0:00:48 


0:02:00 


0:02:41 


0:01:22 


0:02:00 


t 


0:02:35 


0:01:59 


0:02:09 


0:01:53 


WPQ 


0:00:43 


0:01:00 


0:01:00 


0:01:13 


0:01:03 


0:01:00 


t 


0:01:03 


0:01:36 


0:00:47 


0:01:03 


PAQ 


0:01:06 


0:01:02 


0:01:00 


0:00:40 


0:00:44 


0:01:00 


t 


0:00:51 


0:00:45 


0:00:42 


0:00:52 


CMQ 


0:03:52 


0:02:47 


0:01:00 


0:02:36 


0:03:06 


0:02:35 


0:00:09 


0:03:53 


0:03:51 


0:02:14 


0:02:53 


Mean time 


t 


t 


t 


t 


t 


t 


t 


t 


t 


0:39:17 



f Not applicable. 

1 See exhibit 2-1 for full section names. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 fifth-grade data collection, school year 2002-03 timing study. 



Overall, the parent interview required just under 40 minutes to complete. The timings for the 
first interview in the household ranged from a low of 30 minutes to a high of 53 minutes. The approaches 
used to capture the information (update versus obtain new data) and the characteristics of the child and 
household contributed to the variations in the length of interviews. The twin interview required an 
additional 14 minutes. 



No individual section was unduly long. The section that required the most time to administer 
was the Home Environment, Activities and Cognitive Stimulation (HEQ) section. In the first interview 
for respondents (i.e., for child 1), it took between 6 minutes and 14 minutes, depending upon the 
interviewer. Only two respondents completed the Nonresident Parent section of the questionnaire, which 
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included items about parents who did not live with their child, such as a question about the frequency of 
their contact with the child. The average time for this section was about 3 minutes. The results of the 
timing study suggested that the parent questionnaire could be administered within 40 minutes. 



2,4 Child Food Consumption Questions 

To measure children’s food consumption the ECLS-K assessors administered the Food 
Consumption Questionnaire (FCQ), a questionnaire used to determine the kinds of food the children can 
buy at school and food they have eaten in the past week. The FCQ for children consisted of 19 questions. 
There were also food consumption questions for school administrators. Those are described in section 2.6 
below. 



In the FCQ for children, the first set of questions was about foods that are high in fat, 
sodium, and/or added sugars (e.g., candy, salty snacks, soda pop). Children were asked if they could buy 
these foods at school, and if so, how often they bought the food in the past week and where they bought 
the food (vending machine, cafeteria, or somewhere else in school). In the second set of questions, 
children were asked about whether they ate particular key foods and beverages in the past 7 days, such as 
milk, sweetened beverages (e.g., soft drinks), fruits and vegetables, and fast food. They were asked to 
include food they ate at home, at school, at restaurants, or anywhere else. 

Items for the FCQ were taken mainly from existing surveys, although some were developed 
for the ECFS-K. Two main sources for questions were two surveys by the Center for Disease 
Control/Division of Adolescent and School Health Surveys: the Youth Risk Behavior Surveillance 
Survey (YRBSS) and the School Health Programs and Policies Survey (SHPPS). 1 The question on fast- 
food meals was taken from the California Children’s Healthy Eating and Exercise Practices Survey 
(CalCheeps). Questions on soft drinks and children’s at-school consumption of snack foods were 
developed at the U.S. Department of Agriculture (USDA), using YRBSS and CalCheeps questions as 
models. 



Assessors read each question of the FCQ to the child, along with the response categories, 
and the child circled his or her answer. The child was asked to tell the assessor what he or she circled so 
the assessor could enter the answer into the computer. At the beginning of the FCQ, there is an example 

1 Information on these CDC surveys is available at http://www.cdc.gov/nccdphp/dash/ 
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question to show the child what kinds of questions would be asked. The example was also used to show 
the child how to circle a response and to practice telling the assessor what answer had been chosen. After 
the first few questions of the FCQ, if the child appeared to understand the response categories and was in 
one of the higher reading categories in the reading assessment, the child was allowed to read the response 
categories if he or she wanted to do so. 

For children who were homeschooled by their parents or another adult and did not attend 
school, questions about food that could be purchased at school did not apply. For these cases, assessors 
were told to skip questions 1 through 9 and enter “Don’t Know” into CAPI for each of these questions 
and then begin with the statement after question 9. 



2.5 Modifications to Teacher Questionnaires 

The approach for administering teacher questionnaires in spring-fifth grade differed from 
that of previous rounds because many fifth-grade children were expected to have different teachers for 
different subject areas. In the prior rounds of data collection, general education teacher questionnaires 
were designed for a single classroom teacher. All questions pertaining to the core academic subjects were 
asked in a single questionnaire and distributed to one teacher for each sampled child. When children had 
different subject matter teachers, it was left to the child’s main teacher to ask the other teachers to 
complete specific sections of the questionnaire for the subjects they taught. However, as children move 
through the elementary grades, the prevalence of children being taught core academic subjects 
(reading/language arts, mathematics, and science) by a single teacher decreased. Data from the 1999-2000 
Schools and Staffing Survey (SASS) indicated that the proportion of children taught core academic 
subjects by different teachers (i.e., team teaching and departmentalized instruction) was about 41 percent 
in public schools and about 56 percent in private schools. Thus, data collection procedures for spring-fifth 
grade were designed to ensure that the teachers most knowledgeable of the child’s performance in each of 
the core academic subjects (i.e., reading/language arts, mathematics, and science) provided the data 
germane to each child’s classroom environment, instruction in each of the core academic subjects, and the 
core academic teacher’s professional background. 

During the spring-fifth grade data collection, each child’s reading and math or science 
teacher received a self-administered teacher-level questionnaire about a variety of topics, including 
instructional practices, classroom resources, views on teaching and the school, and teacher background. 
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Three additional questionnaires specifically about the focal child were also distributed for teachers in 
reading, math, and science. Each reading and math or science teacher received a teacher questionnaire in 
addition to at least one child-level questionnaire in reading, mathematics, or science. 

All students were assigned to have their reading teacher complete questionnaires. To reduce 
respondent burden for teachers, half of students were randomly assigned to have a mathematics teacher 
complete questionnaires, and half of students were randomly assigned to have a science teacher complete 
questionnaires. In some schools, the sampled children were taught reading, mathematics, and science by 
the same person in one classroom. In other schools, different teachers taught these subjects to the sampled 
children. During the fifth grade data collection, 53 percent of the students were taught in self-contained 
classrooms, 24 percent received departmentalized instruction, 1 8 percent were team taught, and 5 percent 
were pulled out of class for instruction. For more information about how questionnaires were distributed, 
see section 4.5.5. 

The reading teacher questionnaire had three different parts. The first part included questions 
from the Social Rating Scale (SRS) that collected data on five areas of children’s social skills. The second 
part had questions from the Academic Rating Scale (ARS) and gathered data on each sampled child’s 
skills in areas of language and literacy. The third part asked child-specific instructional information (for 
example, child’s grade, additional tutoring or services the child received), asked the teacher to tell how 
this child behaved and performed in language and literacy relative to the other children in the class, and 
asked about the teacher’s classroom and the characteristics of the students, instructional activities and 
curricular focus, and instructional practices in language arts. The mathematics teacher questionnaire 
included questions from the ARS gathering data on each sampled child’s skills in mathematics, asked 
child-specific specific instructional information (for example, child’s grade, additional tutoring or 
services the child received), asked the teacher to tell how this child behaved and performed in 
mathematics class relative to the other children in the class, and asked about the teacher’s classroom and 
the characteristics of the students, instructional activities and curricular focus, and instructional practices 
in mathematics. The science teacher questionnaire was similar to the mathematics teacher questionnaire 
with the questions focusing on science rather than mathematics. Teachers responded to two of these 
questionnaires for each sampled child. Therefore, data were gathered on each sampled child’s skills in the 
areas of language and literacy and mathematical thinking, or in the areas of language and literacy and 
science. For more information on data collection with teachers, see section 4.5.5. 
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In addition to the teacher questionnaires described above, the ECLS-K also included special 
education teacher questionnaires. These were similar to the ones given to special education teachers in 
previous rounds and had two parts, A and B. Part A of the special education teacher questionnaire was 
designed to collect information about the special education teacher’s professional background and 
experience. Part B asked about the special education services provided to the child and the nature of the 
child’s special education curriculum. Except for one change, the spring- fifth grade special education 
teacher questionnaires were identical to the ones used in spring-third grade. A question on the receipt of 
special education or related services due to an attention deficit/hyperactivity disorder (ADHD) was added 
to Part B in the spring-fifth grade questionnaire. 



2.6 Modifications to School Administrator Questionnaire 

The principal, administrator, or headmaster at the school attended by the sampled child was 
asked to complete the school administrator questionnaire in the spring of 2004. As in previous rounds of 
the study, this self-administered questionnaire was intended to gather information about the school, 
student body, teachers, school policies, and administrator characteristics. 

Two main changes were made to the questionnaire in spring-fifth grade. First, it included 
items that in third grade had been in a questionnaire called the school fact sheet (e.g., the grades taught in 
the school, school sector and focus, the length of the school year). Also, a new content area about student 
food consumption was added. The main purpose of these questions was to determine the availability at 
school of various foods, including those that are healthy and those that are high in fat, sodium, and/or 
added sugars. Questions were asked about whether students could purchase food or beverages from 
vending machines at the school or a school store, canteen, or snack bar. School administrators were also 
asked if the school offered a la carte lunch or breakfast items to students that were not sold as part of the 
National School Lunch or the School Breakfast Program. In addition, questions were asked about whether 
children could buy particular foods and beverages at school, such as milk, sweetened beverages (e.g., soft 
drinks), fruits and vegetables, candy, and salty snacks; where these foods could be obtained in the school 
(e.g., a school store, a vending machine); and how full the cafeteria was at peak meal times. Questions on 
the availability of foods not part of USDA meal programs and cafeteria crowding were taken from 
SHPPS. The sources for the other food consumption questions in the school administrator questionnaire 
are the same as those described in section 2.4 about the children’s food consumption questions. 
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3. SAMPLE DESIGN AND IMPLEMENTATION 



This chapter describes the sample design of the Early Childhood Longitudinal Study, 
Kindergarten Class of 1998-99 (ECLS-K), and how it was modified and implemented for each round of 
data collection. An overview of the sample design is given here and described in more detail in the 
following sections. 

The ECLS-K employed a multistage probability sample design to select a nationally 
representative sample of children attending kindergarten in 1998-99. In the base year the primary 
sampling units (PSUs) were geographic areas consisting of counties or groups of counties. The second- 
stage units were schools within sampled PSUs. The third- and final-stage units were students within 
schools. During the base year, data were collected in both the fall and the spring. 

Base year respondents were eligible for the first-grade data collection and nonrespondents 
were not eligible. A case was considered responding for the base year if there was a completed child 
assessment or parent interview in fall- or spring-kindergarten. A child with a disability who could not be 
assessed was also considered a base year respondent whether or not this child had a complete parent 
interview. Background characteristics such as sex, race/ethnicity, age, height, and weight are available for 
children with disabilities who could not be assessed. While all base year respondents were eligible for the 
spring-first grade data collection, fall-first grade was limited to a 30 percent subsample. The spring-first 
grade student sample was freshened to include current first-graders who had not been enrolled in 
kindergarten in 1998-99 and, therefore, had had no chance of being included in the ECLS-K base year 
kindergarten sample. For both fall- and spring-first grade, approximately 50 percent of sampled students 
who had transferred from their kindergarten schools were followed. 

The third-grade data collection included base year respondents and children sampled in first 
grade through a freshening operation in which the spring-first grade sample was freshened to include 
first-graders who had not been enrolled in kindergarten in 1998-99 and therefore had no chance of being 
included in the ECLS-K base year kindergarten sample. As in the first-grade data collection, where only a 
subsample of students who had transferred from their kindergarten schools was followed, subsampling of 
the movers was also used in third grade. In third grade, however, the subsampling rate applied to movers 
was slightly higher: children whose home language was non-English (also known as children belonging 
to the language minority group) and who had transferred for the first time between kindergarten or first 
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grade to third grade were followed with certainty. In other words, 1 00 percent of the children belonging to 
the language minority group who had not moved between kindergarten and first grade but had moved 
between first grade and third grade were followed into their new third-grade schools. Language minority 
children who had moved between kindergarten and first grade and were not subsampled for followup in 
first grade did not re-enter the third-grade sample; those who were subsampled for followup in first grade 
were followed with certainty into their third-grade schools if they had moved again between first grade 
and third grade. The higher subsampling rate allowed for the preservation of this group in the sample for 
analytic reasons. Children not in the language minority group continued to be subsampled for followup at 
a 50 percent rate if they had moved out of the original sample schools. 

In fifth grade, the sample that was fielded was reduced by excluding certain special groups 
of children from data collection, and by setting differential sampling rates for movers in different 
categories. Specifically, children in four groups were not fielded for the fifth-grade survey, irrespective of 
other subsampling procedures that were implemented. These were children who had become ineligible in 
an earlier round because they had died or moved out of the country, children who were subsampled out in 
previous rounds because they were movers, children whose parents emphatically refused to cooperate 
(hard refusals), and children eligible for the third-grade data collection for whom there are neither first- 
grade nor third-grade data. Of the remaining children, those who had moved from their original schools 
during fifth grade or earlier were subsampled for followup. Children whose home language was not 
English (language minority) continued to be a special domain of analytic interest, and were subsampled at 
higher rates. Children were subsampled at different rates depending on the longitudinal data available for 
those children. 

The precision requirements and achieved sample sizes for the different waves of data 
collection are discussed in section 3.1. The base year, fall-first grade, spring-first grade, and spring-third 
samples are discussed in section 3.2, 3.3, 3.4 and 3.5, respectively. Sampling issues that were considered 
prior to the fifth-grade data collection are discussed in section 3.6. Section 3.6.3 includes a discussion of 
the characteristics of the fifth-grade sample and those of the children excluded from the fifth-grade data 
collection. 
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3.1 



Precision Requirements and Achieved Sample Sizes 



The ECLS-K is a nationally representative longitudinal survey of children who attended 
kindergarten in 1998-1999, supplemented with children who were in first grade in spring 2000, but were 
not in kindergarten the previous year. Data on these children were collected from a variety of sources at 
two points in the base year (kindergarten in 1998-1999), two points in the 1999-2000 school year (as 
noted earlier, the fall collection was limited to a subsample of children) when most of the children were in 
first grade, in spring of 2002 when most of the children were in third grade, and again in spring of 2004 
when most of the children were in fifth grade. 

The overall design for the survey evolved over time. The initial design study recommended 
sampling 23,500 children in approximately 1,000 kindergarten programs sampled from 100 PSUs. The 
initial plans also called for sampling children in private schools at a higher rate than children in public 
schools, as well as sampling minorities (children of Black, Hispanic, or Asian or Pacific Island [API] race 
or ethnicity) at higher rates than nonminorities. The design study assumed that because of nonresponse 
and losses due to children moving, the final number of completed interviews at the end of the survey 
would be about 10,300. While the design study was useful in providing overall direction, the final 
framework for the sample design differed in many ways from its recommendations. 

The sample design implemented through the fifth grade in the ECLS-K is described in this 
chapter. The remainder of this section gives an overview of the sampling objectives and how the design 
was revised to accommodate changes in those objectives over the course of the study. Subsequent 
sections of the chapter give the details of the procedures used to implement the sample in the various 
rounds or waves of data collection, beginning with the base year in 1998-1999. 

Four precision requirements for the survey were identified and formed the basis for the base 
year sample design and plans for the followups in subsequent rounds. These requirements are the ability 
to do the following: 



■ Measure a relative change of 20 percent in proportions across waves; 

■ Measure a relative change of 5 percent in a mean assessment score across waves; 



3-3 




Estimate a proportion for each wave with a coefficient of variation (CV) of 1 0 percent 
or less; and 



■ Estimate a mean assessment score for each wave with a CV of 2.5 percent or less. 

The goals were interpreted as being objectives not only for all children, but for subgroups of 
analytic interest that include children attending public and private schools (Catholic, non-Catholic), and 
children from different race and ethnic groups (Hispanic, Black, Asian and Pacific Islander, all other 
races). After the spring-first grade data collection, language minority children were a newly identified 
subgroup of analytic interest for sample design purposes. A large number of assumptions had to be made 
to estimate sample sizes sufficient to meet the precision requirements. The key assumptions included 
projections of the losses due to nonresponse and attrition due to children moving, the design effects 1 
associated with the sample design, the element mean and standard deviations of the assessment scores, 
and the correlation of the statistics across waves. Since the ECLS-K is the first study of this population 
using this methodology, many of the assumptions had to be based on judgments without much supporting 
empirical data. 

The precision requirements that drive the sample design (those demanding the largest sample 
size) have to do with estimating changes over time and estimating the precision of estimates in the fifth- 
grade data collection. Based on assumptions described above, it was determined that a sample in fifth 
grade of about 1 0,000 children would be adequate to meet the precision requirements overall and for most 
subgroups. A sample of about 800 to 1,000 children in a subgroup would be achieved for most of the 
subgroups with an overall sample of 10,000 children and these would approximately meet the precision 
goals. For example, with a sample size of 10,000, the number of Hispanic and Black children would 
exceed 1,000, as shown in section 3.6.3. Children in private schools and APIs were the two subgroups 
that were expected to fall short of the goals if higher sampling rates were not applied. As noted in the 
following sections, sampling procedures were implemented to increase the sample size for these two 
groups. 



After the spring-first grade data collection was completed, the assumptions were reviewed 
and the ability of the sample to meet the survey goals was re-examined. At that time, language minority 
children were included as a subgroup of analytic interest. The evaluation showed that the sample sizes 
were adequate for most subgroups, but special efforts were needed to retain language minority children in 
subsequent rounds. Table 3-11 in section 3.5 shows the outcome of the spring-first grade data collection 

1 When a clustered sample with unequal sampling weights is used, the estimates are less precise than those expected from a simple random 
sample and the ratio of the actual to simple random sampling variance is called a design effect. 
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by type of children. Since funding was made available to support these efforts, sampling procedures for 
retaining movers were modified. In the first-grade data collection, half of the movers were subsampled 
and included for followup, without taking any characteristics of the children into account. To increase the 
sample of language minority children, the sampling procedures were revised for the third-grade followup 
to retain as many of these children as possible. 

The evaluation also showed that the assumed design effects for assessment scores (reading, 
math and general knowledge) were larger than originally expected, ranging from 4.5 to 9.5. The larger 
than expected design effects for scores were first identified after the base year. The design effects for 
percentages, ranging from 1.6 to 6.9 for proportions greater than 30 percent, were close to those originally 
anticipated (3.8 on average). 2 The evaluation showed that the correlation over time of the scores was 
higher than expected. The higher correlation makes estimates of change in scores over time more precise. 
Consequently, the only precision objective that is substantially affected by the higher than expected 
design effects is for the mean assessment scores for fifth grade. This partially offsets the loss in precision 
due to the higher design effect. 

Table 3-1 tracks the ECLS-K sample from the base year through fifth grade. The table shows 
that the large initial sample of children has been reduced over time due to subsampling movers and 
nonresponse, as expected. While the initial assumptions that drove the sample design were not always 
accurate separately, the overall effect of the losses has been very close to what was expected. For 
example, in several rounds of the ECLS-K, the assumed moving rate was lower than the actual moving 
rate, but this was offset by higher completion rates. The overall number of eligible children at the end of 
the fifth-grade wave was more than 12,000 children, and the final sample size for the fifth-grade sample 
exceeded the 10,000 children in the initial projections. 



2 See design effects for selected survey items in chapter 4 of the ECLS-K Base Year Public-Use Data Files and Electronic Code Book: User’s 
Manual (NCES 2001-029r) (Tourangeau, Burke, et al. 2004). 
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Table 3-1. ECLS-K sample size from the base year through fifth grade: School years 1998-99, 
1999-2000, 2001-02, and 2003-04 



Characteristic 


Fall- 

kindergarten 


Spring- 

kindergarten 


Fall- 

first 

grade 1 


Spring- 

first 

grade 


Spring- 

third 

grade 


Spring- 

fifth 

grade 


Initial sample 


21,387 


22,772 2 


6,507 


21,357 3 


21,357 3 


16,143 4 


Fielded after subsampling movers 


t 


t 


5,728 


18,507 


17,240 


12,380 


Fielded after locating movers 


t 


22,088 


5,691 


17,708 


16,951 


12,170 


Number of eligibles 


21,356 


21,941 


5,652 


17,652 


16,829 


12,129 


Child-complete 5 


19,173 


19,967 


5,291 


16,727 


14,470 


11,346 


Parent-complete 6 


18,097 


18,950 


5,071 


15,626 


13,489 




Child- or Parent-complete 


19,864 


20,578 


5,424 


17,324 


15,305 


11,820 


Child- and Parent-complete 


17,586 


18,339 


4,938 


15,029 


12,654 





f Not applicable. 

1 Only 30 percent of base year schools were included in the fall-first grade sample. 

2 Including 1,426 students from refusal converted schools and excluding 41 students in schools that cooperated in Fall-Kindergarten and refused 
in Spring-Kindergarten. 

3 Only students who have at least one of the four base year data points (fall-kindergarten assessment or parent data, or spring-kindergarten 
assessment or parent data, and the 1 65 students sampled in first grade through sample freshening. 

4 Excluding students described in section 3.6.1. 

5 Child-complete if the child had assessment data or was not assessed due to a disability. 

6 Parent-complete if the child had parent interview data. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), fall 1998, spring 1999, spring 2000, spring 2002, and spring 2004. 



The details on the sample sizes for subgroups at the end of the fifth grade are provided later 
in this chapter (see tables 3-20, 3-21, and 3-22). Those tabulations show that the number of fifth-grade 
respondents for all of the specific subgroups of interest exceeds 1,000, except for children in non-Catholic 
private schools and API children. For most of the key analytic groups the numbers of respondents are 
much larger than 1,000. For API children, the number of respondents is 970, which exceeds the minimum 
target of 800 and is very close to 1,000. The number of respondents in non-Catholic private schools is 
957. 



3.2 Base Year Sample 

In the base year, the ECLS-K selected a nationally representative sample of children 
attending kindergarten in 1998-99, using a dual-frame multistage probability sample design. Counties 
and groups of counties constituted the first-stage sampling units or PSUs, schools or kindergarten 
programs within PSUs were the second-stage units, and children were the third- and final-stage units. 
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3.2.1 



Selecting the Area Sample 



The point of departure for the ECLS-K area sample frame development was an existing 
multipurpose frame of PSUs created, using 1990 county-level population data and 1988 per capita income 
data from the U.S. Department of Commerce, Bureau of Economic Analysis. This frame contained 1,404 
PSUs that were counties or groups of contiguous counties. PSUs did not cut across census regional 3 
boundaries, but were allowed to cross state boundaries. Each 1990 metropolitan statistical area (MSA) 4 
constituted a single PSU except where an MSA crossed census regions, and it was split into two PSUs. 
The minimum size of a PSU in the multipurpose frame was 15,000 persons. 

Since the focus of the ECLS-K is kindergarten students, the existing PSU frame was updated 
with 1994 population estimates of 5-year-olds by race/ethnicity, the most up-to-date estimates available 
from the U.S. Bureau of the Census at the time. The counts of 5-year-olds by race/ethnicity were used to 
revise PSU definitions relative to a different minimum PSU size and to construct a measure of size 
(MOS) that facilitated the oversampling of APIs. Each PSU in the frame that did not have at least 320 
5-year-olds was collapsed with an adjacent PSU. This minimum PSU size was developed based on 
assumptions concerning anticipated school response rates, the average number of schools that would be 
selected per PSU, and the target number of students to be sampled per school. After this collapsing, the 
final ECLS-K PSU frame contained 1,335 records. 

The MOS used for selecting PSUs took into account the amount of oversampling of APIs 
required to meet the ECLS-K precision goals. The weighted MOS was calculated as follows: 



MOS = 2.5 x n API + n other 



where 2.5 is the oversampling rate for APIs, and n API and n other are the counts of 5-year-old APIs and all 
others, respectively. The oversampling rate for APIs was calculated as the target number of completed 
API cases divided by the expected number of completed API cases without oversampling. In all, 100 
PSUs were selected for the ECLS-K. The 24 PSUs with the largest measures of size were designated as 
certainty selections or self-representing (SR) 5 and were set aside. They were included in the sample with 
certainty. Once the self-representing PSUs were removed, the remaining PSUs, called non-self- 



3 A census region is a geographic region defined by the U.S. Bureau of the Census. 

4 A metropolitan statistical area (MSA) is a geographic entity designated as one or more counties in a metropolitan area, except in New England, 
where MSA is defined in terms of county subdivisions. MSAs generally have under 1 million in population. 

5 A self-representing PSU is selected into the sample with certainty (i.e., with probability 1). 
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representing (non-SR) 6 , were partitioned into 38 strata of roughly equal MOS. The frame of non-self- 
representing PSUs was first sorted into eight superstrata by crossing the two MSA categories (MSA and 
non-MSA) and the four census region (Northeast, Midwest, South, and West). Within the four MSA 
superstrata, the variables used for further stratification were race/ethnicity (high concentration of API, 
Black, or Hispanic), size class (MOS > 13,000 and MOS < 13,000), and 1988 per capita income range 
(shown in table 3-2, each range was defined so as to have roughly equal population in each of the stratum, 
where applicable) Within the four non-MSA superstrata, the stratification variables were race/ethnicity 
and per capita income. The term “superstrata” is used here to distinguish between the larger strata created 
by crossing MSA categories and census regions and the smaller strata defined by race/ethnicity, size class 
and per capita income. Table 3-2 describes how the 38 non-self-representing strata were created. 

Two PSUs were selected from each non-self-representing stratum using Durbin’s Method 
(Durbin 1967). This method selects two first-stage units per stratum without replacement, with probability 
proportional to size and a known joint probability of inclusion. The Durbin method was used because it 
has statistical properties that make it easier to compute variances. Table 3-3 summarizes the 
characteristics of the ECLS-K PSU sample. 

The Durbin method required two passes of the frame with a different selection probability at 
each pass to obtain the desired probabilities of inclusion and joint probabilities of inclusion. In the first 
pass, one PSU was selected in the stratum with probability p In the second pass, the selected PSU was 

excluded and another PSU was selected with probability proportional to 

1 1 

pi 1 

L'-^l 1-2R2 J 

where /q = M^/M and p 1 = My'/ff, M | is the MOS of the first unit selected, M 1 the MOS of the second 
unit selected, and M the MOS of the stratum. 



The overall selection probability of non-self-representing unit i is 



Pi = 



2 Mi 

M ’ 



i = 1, 2 . 



6 A non-self-representing PSU is selected into the sample with probability proportional to its measure of size (MOS). 
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Table 3-2. Stratum definitions for the 38 non-self-representing strata: School year 1998-99 



Stratum 


Metropolitan 
statistical area 
(MSA) status 1 


Census 

region 2 


Race/ethnicity 
(percentage range) 


PSU 3 measure _ 


Per capita income range 


of size (MOS) 


Low 


High 


1 


MSA 


N ortheast 


Any 


> 13,000 


$22,062 


$25,424 


2 


MSA 


Northeast 


Any 


> 13,000 


16,342 


22,030 


3 


MSA 


Northeast 


Any 


< 13,000 


18,128 


29,084 


4 


MSA 


Northeast 


Any 


< 13,000 


16,697 


18,032 


5 


MSA 


N ortheast 


Any 


< 13,000 


12,279 


16,616 


6 


MSA 


Midwest 


Any 


> 13,000 


17,277 


18,150 


7 


MSA 


Midwest 


Any 


> 13,000 


16,103 


17,092 


8 


MSA 


Midwest 


Any 


< 13,000 


16,552 


24,009 


9 


MSA 


Midwest 


Any 


< 13,000 


15,732 


16,475 




MSA 


Midwest 


Any 


< 13,000 


14,450 


15,693 


11 


MSA 


Midwest 


Any 


< 13,000 


10,185 


14,433 


12 


MSA 


South 


Hispanic >30 


Any 


Any 


Any 


13 


MSA 


South 


Black > 40 


Any 


Any 


Any 


14 


MSA 


South 


26 < Black < 40 


Any 


14,743 


18,731 


15 


MSA 


South 


26 < Black < 40 


Any 


10,892 


14,573 


16 


MSA 


South 


Black < 26 


> 13,000 


16,435 


16,601 


17 


MSA 


South 


Black < 26 


> 13,000 


14,586 


16,337 


18 


MSA 


South 


Black < 26 


< 13,000 


15,572 


22,824 


19 


MSA 


South 


Black < 26 


< 13,000 


14,194 


15,432 


20 


MSA 


South 


Black < 26 


< 13,000 


11,262 


13,979 



See notes at end of table. 
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Table 3-2. Stratum definitions for the 38 non-self-representing strata: School year 1998-99 — Continued 



Stratum 


Metropolitan 
statistical area 
(MSA) status 1 


Census 

region 2 


Race/ethnicity 
(percentage range) 


PQT f mpcieiirp __ 


Per capita income range 


roU IllCUbUIC — 

of size (MOS) 


Low 


High 


21 


MSA 


West 


Asian/Pacific Islander >15 


Any 


Any 


Any 


22 


MSA 


West 


Asian/Pacific Islander >15 


Any 


Any 


Any 


23 


MSA 


West 


Hispanic >30 


Any 


Any 


Any 


24 


MSA 


West 


12 < Hispanic < 30 


Any 


Any 


Any 


25 


MSA 


West 


Hispanic <12 


Any 


15,048 


21,840 


26 


MSA 


West 


Any 


Any 


9,993 


14,839 


27 


Non-MSA 


Northeast 


Any 


Any 


Any 


Any 


28 


Non-MSA 


Midwest 


Any 


Any 


14,124 


17,446 


29 


Non-MSA 


Midwest 


Any 


Any 


13,277 


14,121 


30 


Non-MSA 


Midwest 


Any 


Any 


12,169 


13,272 


31 


Non-MSA 


Midwest 


Any 


Any 


6,992 


12,147 


32 


Non-MSA 


South 


Black > 42 


Any 


Any 


Any 


33 


Non-MSA 


South 


25 < Black < 42 


Any 


Any 


Any 


34 


Non-MSA 


South 


Any 


Any 


12,727 


20,059 


35 


Non-MSA 


South 


Black < 25 


Any 


11,165 


12,676 


36 


Non-MSA 


South 


Any 


Any 


6,018 


11,142 


37 


Non-MSA 


West 


Any 


Any 


12,887 


23,286 


38 


Non-MSA 


West 


Any 


Any 


6,959 


12,884 



1 MSA is a geographic entity designated as one or more counties in a metropolitan area, except in New England, where MSA is defined in terms 
of county subdivisions. Non-MSA designates one or more counties not in a metropolitan area. MSA and non-MSA are as defined by the Bureau 
of the Census. 



2 A census region is a geographic region defined by the U.S. Bureau of the Census. 

3 Primary sampling unit. 

NOTE: In this table, “Any” means any value of the column variable. For example, stratum 1 includes PSUs that have MSA status, are located in 
the Northeast region, with a MOS greater than or equal to 13,000 and per capita income ranging between $22,062 and $25,424, and can have any 
value of the race/ethnicity percentage. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), fall 1998 and spring 1999. 



The joint probability of inclusion of the first and second units is 



n \,2 



2 P\Pl 
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Table 3-3. Distribution of the ECLS-K primary sampling unit (PSU) sample by self-representing (SR) 
status, metropolitan statistical area (MSA) status, and census region: School year 1998-99 



SR status 


MSA status 






Census region 1 




Total 


Northeast 


Midwest 


South 


West 


Total 




100 


18 


25 


34 


23 


SR 


MSA 


24 


6 


5 


6 


7 


Non-SR 


MSA 


52 


10 


12 


18 


12 


Non-SR 


Non-MSA 


24 


2 


8 


10 


4 



1 A census region is a geographic region defined by the U.S. Bureau of the Census. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), fall 1998 and spring 1999. 



3.2.2 Selecting the School Sample 

In the second stage of sampling, public and private schools offering kindergarten programs 
were selected. For each ECLS-K PSU, a frame of public and private schools offering kindergarten 
programs was constructed, using existing school universe files: the 1995-96 Co mm on Core of Data 
(CCD) (U.S. Department of Education 1995-96) and the 1995-96 Private School Universe Survey (PSS) 
(Broughman and Colaciello 1998). The school frame was freshened in the spring of 1998 to include 
newly opened schools that were not included in the CCD and PSS and schools that were in the CCD and 
PSS but did not offer kindergarten, according to those sources. A school sample supplement was selected 
from the supplemental frame. 



3.2.2. 1 School Frame Construction 

The 1995-96 CCD Public School Universe File was the primary source for the ECLS-K 
public school sampling frame. Most schools run by the U.S. Department of the Interior, Bureau of Indian 
Affairs (B1A) and the schools run by the U.S. Department of Defense (DOD) were not included on the 
1995-96 CCD. The 1995-96 Office of Indian Education Programs Education Directory (U.S. Department 
of the Interior, Bureau of Indian Affairs unpublished document) was consulted, in order to complete the 
list of B1A schools in the CCD file. For the DOD schools, a 1996 list of schools obtained directly from 
the DOD was used. The 1995-96 PSS Universe File was used as the primary source of the private school 
sampling frame. 
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The first step in frame construction involved subsetting the file to schools located in counties 
that constituted the ECLS-K PSU sample. Further subsetting retained only those schools that offered 
transitional kindergarten, kindergarten, or transitional first grade, or which were strictly ungraded, as 
indicated by the school’s grade span. 

The constructed ECLS-K school frame included 18,911 public-school records and 12,412 
private-school records. This frame constituted the original frame. The original frame was supplemented in 
the spring of 1 998 to include schools that would be operational in fall 1 998 but had not been included in 
the original frame. The procedures used to construct the supplemental or freshened frame are given later 
in this section. 



Table 3-4 gives the estimated number of schools offering kindergarten programs and the 
number of kindergarten students from the ECLS-K school frame. These are the numbers of schools and 
students in the sampled PSUs in the frame weighted by the inverse of the PSU selection probabilities. 



Table 3-4. Estimates of the number of kindergarten schools and students, by primary sampling unit 
(PSU) status: School year 1998-99 



Estimated number of 
kindergarten schools 


Estimated number of 
kindergarten students 




Total 


Public 


Private 


Total 


Public 


Private 


Total 


73,095 


50,084 


23,011 


4,089,781 


3,521,040 


568,741 


Self-representing PSUs 
Non-self-representing PSUs 


19,721 

53,374 


11,283 

38,801 


8,438 

14,573 


1,277,419 

2,812,362 


1,059,535 

2,461,505 


217,884 

350,857 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), fall 1998 and spring 1999. 



3.2.2.2 School Measure of Size 

Within each PSU, schools with fewer than a predetermined minimum number of 
kindergarten students were clustered together before sampling in order to obtain a sample that is closer to 
self-weighting. The minimum number of kindergartners was 24 for public schools and 12 for private 
schools. Schools were selected with probability proportional to size. As with the PSU sample, a weighted 
MOS was constructed taking into account the oversampling of APIs: 
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SCHMOSy 2.5 x n AP /y + n other, ij 



where 2.5 is the oversampling rate for APIs, and n APIi j and n 0( / ier are the counts of API kindergarten 
students and all other kindergarten students, respectively, in school j of PSU i. 



3.2.2.3 School Allocation 

Schools were sampled at rates designed to result in an approximately self-weighting sample 
of students within public and private school strata. The target number of sampled schools per PSU was 
calculated separately for public schools and private schools, and for self-representing and non-self- 
representing PSUs. The number of schools selected was the target number of schools adjusted upward by 
the estimated school response and eligibility rate. 



3.2.2.3.1 Public Schools 

The total MOS for public schools was partitioned into the self-representing and non-self- 
representing strata. There are 100 PSUs in the ECLS-K sample, of which 24 are in the self-representing 
strata. The number of public schools selected from the self-representing strata was calculated as 

24 

X w f x PSU MOS i 

n SR=W) x « 

X n’ ( x PSUMOSi 
i=l 

where n is the total number of public schools to be selected, tv, is the weight of PSU i, and 

PSUMOSi = X SCHMOSy . 
j 

The value for n is 800/. 85 = 941 where .85 is the expected eligibility and response rate for 
public schools. The supplement of public schools was expected to add relatively few schools to the frame 
and thus the 85 percent rate was not modified. The distribution of sampled schools was approximately 
291 for self-representing strata and 650 for non-self-representing strata. For self-representing and non- 
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self-representing strata alike, the number of schools allocated to each PSU was proportional to the 
weighted MOS of the PSU ( WixPSUMOS ,). 



In the ECLS-K public school frame, 4 percent of public schools had fewer than 24 
kindergarten students. These schools were combined with other schools in the same PSU to form clusters 
with at least 24 students prior to sampling. Schools with 24 students or more were not grouped, but were 
also referred to as clusters (of one school each). To sample approximately 941 public schools, around 915 
clusters (single schools or groups of schools) have to be selected. As a general rule, if a sampled school or 
cluster of schools had 24 or more students, 24 students were selected. However, for practical reasons, all 
students in the sampled school or cluster were selected if there were fewer than 27 students. More details 
on the clustering of schools are found in the next section. 



The number of clusters was allocated to each PSU proportionally to the weighted MOS of 
the PSU ( WjxPSUMOS ',-). When the 915 clusters were allocated to PSUs, it was discovered that in 5 PSUs 
there were not enough clusters in the frame to select the required number of clusters. As a result, only 900 
clusters were selected. Table 3-5 shows the expected distributions of clusters, schools, and students. 



Table 3-5. Expected number of clusters, schools, and students — public schools: School year 1998-99 



Type of primary sampling 
unit (PSU) 


Number of 
clusters to 
select 


Expected 
number of 
schools sampled 


Expected 
number of 
students sampled 


Average 
number of 
students/school 


Total 


900 


944 


21,643 


23 


Self-representing PSUs 


283 


285 


6,792 


24 


Non-self-representing PSUs 


617 


659 


14,851 


23 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class 
of 1 998-99 (ECLS-K), fall 1 998 and spring 1 999. 



3.2.2.3.2 Private Schools 

The procedure used to determine the allocation of the public schools was also used for 
allocating the private schools. The private school target samples are labeled nsR' and nj\fSR' for self- 
representing and non-self-representing PSUs respectively, and n' is the sum of hsr' and n,\jSR' ■ The value 
of n' is 200/.60=333, where .60 is the expected eligibility and response rate. The supplement to the frame 
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was expected to add some private schools with kindergarten programs. The 60 percent rate was used 
because of the uncertainties associated with the estimate of the eligibility and response rate for private 
schools. 



The percentage of schools with fewer than 24 kindergarten students was large for private 
schools. Approximately 56 percent of private schools offered a kindergarten program that had fewer than 
24 students, and 44 percent of these small schools have fewer than 12 students in their kindergarten 
program. Schools having fewer than 12 kindergarten students (according to the frame) were grouped into 
clusters of schools with at least 12 students in each cluster, following the clustering rules discussed in the 
next section. Schools with 12 students or more were not grouped. As a general rule, if a sampled school or 
cluster of schools had 24 or more students, 24 students were selected; if a sampled school or cluster had 
fewer than 24, all students were sampled. However, for practical reasons, all students in the sampled 
school or cluster were selected if there were fewer than 27 students. 

In order to sample approximately 333 private schools, 278 clusters were selected (single 
schools or groups of schools). Table 3-6 shows the expected distributions of clusters, schools, and 
students. 



The number of clusters was not allocated separately to each self-representing PSU, since 
sampling was done on the aggregated list of school clusters in the self-representing PSUs. This 
aggregated list of school clusters in the self-representing PSUs had been sorted prior to sampling by 
religious affiliation in order to have better control of the sample distribution by religious affiliation. For 
the non-self-representing PSUs, the sample was allocated to each PSU proportionally to the weighted 
MOS of the PSU (wixPSUMOSj), with a minimum of one cluster per PSU imposed if the PSU was so 
small that it was not allocated any clusters. 
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Table 3-6. Expected number of clusters, schools, and students — private schools: School year 1998-99 



Type of primary sampling 
unit (PSU) 


Number of 
clusters to 
select 


Expected 
number of 
schools sampled 


Expected 
number of 
students sampled 


Average 
number of 
students/school 


Total 


278 


333 


6,336 


19 


Self-representing PSUs 


107 


125 


2,456 


20 


Non-self-representing PSUs 


171 


208 


3,880 


19 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), fall 1998 and spring 1999. 



3.2.2.4 Clustering of Small Schools 

As noted above, schools with fewer than 24 students (public) or 12 students (private) were 
clustered together in order to obtain a sample that was closer to self-weighting. For example, if a school 
with 12 students was not clustered, the students from that school would be sampled at about half the 
probability as students in larger schools. The goal of the clustering of small schools was to form school 
clusters with a small number of schools, each cluster having close to 24 students and including 
heterogeneous schools. This goal was set so that if a cluster was selected, it would not be necessary to 
recruit many small schools; furthermore, the heterogeneity of schools improves the reliability of the 
estimates. Heterogeneity was defined by school size for public schools, and by religious affiliation and 
school size for private schools. Within each PSU, schools with fewer than a predetermined minimum 
number of kindergarten students were separated from the frame and clustered together. A few exceptions 
to this general rule did occur and are discussed later. The procedures for clustering of schools are 
described below. 



3.2.2.4.1 Public Schools 

Public schools with fewer than 24 kindergarten students were clustered. Within each PSU, 
the list of small schools (i.e., schools with fewer than 24 kindergartners) was sorted in ascending order of 
kindergarten enrollment; it was then split in half, with the second half re-sorted in descending order. The 
two halves were then put together in an interleaving fashion. Beginning at the top of the list, clusters of 
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schools with at least 24 kindergarten students were formed. If the last cluster on the list still did not have 
the required 24 minimum, then it was put together with the next-to-last cluster on the list. 

This clustering scheme resulted in 18 clusters with 5 or more schools, which were 
considered problematic as far as fieldwork was concerned. The worst case was one cluster with 13 
schools and only 4 1 students. In order to minimize the number of clusters having 5 or more schools, each 
problematic cluster was broken into groups of 2 or 3 schools, and each group was combined with the 
smallest of the “large” schools having 25 or more kindergarten students. Since enrollment in schools with 
missing kindergarten enrollment was imputed to be equal to 24, grouping any of these imputed schools 
with another school was avoided, lest they turn out not to have kindergarten students. 

In addition to the 18 problematic clusters above, there were 12 PSUs with only 1 small 
school (with fewer than 24 kindergarten students) and there were 2 PSUs with only 2 small schools that, 
when grouped together, still had fewer than 24 kindergarten students. These small schools or groups of 
small schools were manually combined with the smallest school in another PSU (not one with only 1 or 2 
schools) having 25 or more students (see table 3-7). 



Table 3-7. Number of clusters and schools in the public school frame: School year 1998-99 



Number in cluster 


Number of clusters 


Number of schools 


Total 


18,399 


18,911 


1 school 


18,095 


18,095 


2 schools 


153 


306 


3 schools 


97 


291 


4 schools 


51 


204 


5 schools 


3 


15 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), fall 1998 and spring 1999. 



3.2.2.4.2 Private Schools 

Private schools with fewer than 12 kindergarten students were clustered. Within each PSU, 
the list of private schools was first sorted by religious and nonreligious affiliation. If the number of 
religious schools and nonreligious schools in the PSU differed by no more than a factor of 3, the smaller 
of the two lists (religious or nonreligious) was sorted in descending order while the larger of the two lists 
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was sorted in ascending order of kindergarten enrollment. The two lists were then put together in an 
interleaving fashion, so that the records that were at the bottom of the longer list were records with larger 
kindergarten enrollment, and did not have to be grouped together. Beginning at the top of the entire list, 
clusters of schools of at least 12 kindergarten students were formed. If the last cluster on the list still did 
not have the required minimum size, it was put together with the next-to-last cluster on the list. 

If the number of religious schools and nonreligious schools in the PSU differed by a factor 
greater than 3, schools were not separated into religious and nonreligious lists. Instead, the entire list of 
schools was sorted in ascending order of kindergarten enrollment; it was then split in half, with the second 
half re-sorted in descending order. The two halves were then put together in an interleaving fashion. 
Clusters of schools were formed as above. 

There were 3 PSUs where the clustering of small schools as specified above did not work 
well. Two of the 3 PSUs had only 1 small school each and the third one had 2 small schools that, when 
grouped together, still had fewer than 12 kindergarten students. These small schools or groups of small 
schools were manually combined with other large schools in another PSU (table 3-8). 



Table 3-8. Number of clusters and schools in the private school frame: School year 1998-99 



Number in cluster 


Number of clusters 


Number of schools 


Total 


9,955 


12,412 


1 school 


7,640 


7,640 


2 schools 


2,184 


4,368 


3 schools 


121 


363 


4 schools 


9 


36 


5 schools 


1 


5 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), fall 1998 and spring 1999. 



3.2.2. 5 Implicit Stratification of Schools/Clusters of Schools 

Public schools with more than 24 kindergarten students and private schools with more than 
12 kindergarten students were not clustered. However, they are referred to as clusters (of one school each) 
for simplicity. 



3-18 







3.2.2. 5.1 Public Schools 



Within each PSU, the clusters were sorted by the MOS and separated into three size classes 
of roughly equal size (high, medium, and low). Within each size class, clusters were sorted by the 
proportion of APIs in a serpentine manner (alternating the sort order from one size class to the next). 



3.2.2. 5.2 Private Schools 

Within each PSU, each cluster was identified as religious, mixed, or nonreligious. 7 The list 
of clusters was then sorted by these three categories. Within each category, the clusters were sorted in a 
serpentine manner by the MOS prior to selection. However, for the self-representing PSUs, all clusters 
were sorted as if they were from the same PSU, i.e., the aggregated list of clusters from the 24 self- 
representing PSUs was sorted by religious affiliation (religious/mixed/nonreligious). This procedure 
provided better control of the sample distribution of religious/mixed/nonreligious clusters. Across non- 
self-representing PSUs, clusters were sorted by religious affiliation, and within each category of religious 
affiliation, by the MOS in a serpentine manner. 



3.2.2. 6 School Selection 

Selection of the clusters of schools was systematic, with probability proportional to the 
MOS. Sampling of public schools was done independently within PSU (i.e., each PSU forms a separate 
sampling stratum) after the clusters of schools were sorted by MOS and proportion of API. Sampling of 
private schools was done separately for self-representing PSUs and for non-self-representing PSUs. All 
self-representing PSUs were placed in one sampling stratum and all non-self-representing PSUs were 
placed in a second stratum. In the self-representing stratum, sampling was done with one random start 
after sorting clusters of schools by religious affiliation and MOS. In the non-self-representing stratum, 
sampling was done with one random start after sorting clusters of schools by PSU, religious affiliation, 
and MOS. 



7 A private school cluster is “religious” if all schools in the cluster are Catholic schools or non-Catholic religious schools; “nonreligious” if all 
schools in the clusters have no religious affiliation; “mixed” if it has a combination of schools with or without religious affiliation. 
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3.2.2. 7 



The ECLS-K Main School Sample 



A total of 1,280 schools were selected from the main school frame for the ECLS-K, of which 
934 were public and 346 were private schools. The characteristics of the school sample are presented in 
table 3-9. 



3.2.2. 8 Supplemental School Sample 

As mentioned earlier, the public and private school frames were supplemented in the spring 
of 1998. The procedures for supplementing the frames were different for public schools, Catholic schools 
and non-Catholic private schools. These procedures are discussed below separately. 



3.2.2. 8.1 Public Schools 

Each public school district having one or more schools sampled was sent a sampling frame- 
based list of all schools offering kindergarten. Districts were asked whether any school that was expected 
to offer kindergarten in academic year 1998-1999 was missing from the list. For each school identified by 
the district, school name, address, telephone number, grade span, and kindergarten enrollment were 
obtained. Districts were also contacted that fell within the boundaries of the ECLS-K PSUs, but for which 
the CCD file listed no schools offering kindergarten, unless it was clear from their name that they were 
strictly secondary school districts (e.g., Middlebury Union High School District). The information 
obtained from the school districts was checked against the ECLS-K public school frame to confirm that 
these schools were truly new or newly eligible. Bona fide new schools were given a chance of being 
sampled. A new school’s chance of selection was conditioned on the school district’s probability of 
selection. Overall, 252 new public schools were identified. Of these, 19 were selected, using systematic 
sampling with probability proportional to size where the MOS was the same as it was for schools sampled 
from the main sample. Thus, a total of 953 public schools were included in the sample (934 + 19). 
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Table 3-9. Number of sample schools, by school characteristics: School year 1998-99 



Characteristic 


Total 


Public 


Private 


Total 


1,280 


934 


346 


Region 


Northeast 


238 


166 


72 


Midwest 


297 


215 


82 


South 


420 


309 


111 


West 


325 


244 


81 


Type of locale 


Large central city 


245 


164 


81 


Mid-size central city 


252 


176 


76 


Urban fringe of large city 


386 


273 


113 


Urban fringe of mid-size city 


98 


78 


20 


Large town 


32 


25 


7 


Small town 


107 


80 


27 


Rural 


160 


138 


22 


Kindergarten enrollment 


<25 


210 


55 


155 


25-49 


224 


110 


114 


50-99 


467 


400 


67 


100-149 


236 


228 


8 


150-199 


88 


86 


2 


200 - 249 


26 


26 


0 


250-299 


15 


15 


0 


>300 


14 


14 


0 


School affiliation 


Public 


934 


934 


t 


Catholic 


117 


t 


117 


Non-Catholic, religious 


143 


t 


143 


Nonreligious, private 


86 


t 


86 


National school lunch program 1 


Low (<=25% eligible students) 


284 


284 


t 


Medium low (>25% and <=50%) 


169 


169 


t 


Medium high (>50% and <=75%) 


122 


122 


t 


High (>75%) 


118 


118 


t 


Unknown 


241 


241 


t 



f Not applicable. 

1 National school lunch program applies only to public schools and hence the counts of schools in the program do not add up to 1,280 schools. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), fall 1998 and spring 1999. 
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3.2.2. 8.2 Private Schools 



The procedure for obtaining new school information from Catholic dioceses was exactly the 
same as for public schools. Since a diocese could cut across county or even state lines, each school 
identified by a diocese had to be associated with the correct county, and hence the correct PSU, before 
checking to see whether it was truly new. Since dioceses might cross PSU boundaries, a new Catholic 
school’s chance of being sampled had to be conditioned on the diocese’s probability of selection within 
the PSU where the new school was located. There were 126 new Catholic schools identified, and 6 were 
selected using systematic sampling with probability proportional to size. When combined with the main 
sample, the final Catholic school sample size was 123 (1 17 + 6). 



3.2.2. 8.3 Non-Catholic Private Schools 

The search for non-Catholic private schools was considerably more complicated. Three 
classes of schools that had previously not been given a chance of selection from the PSS were 
reconsidered. Those were schools that had an u nk nown grade span because they had not responded to the 
1995-96 PSS, those that responded but did not report offering kindergarten, and those that appeared for 
the first time on the 1997-98 PSS file. Together these accounted for 2,544 potential new non-Catholic 
private schools. Beyond these additions from PSS, procedures similar to those used by the Bureau of the 
Census in the PSS area frame search were followed. These procedures included collecting lists of schools 
from different sources, matching them against the PSS list frame to remove duplicates, and further 
screening by telephone to verify new school status. The majority of new schools found by the Bureau of 
the Census for PSS came from telephone book yellow page listings. The yellow pages search was the 
main source of new non-Catholic private schools in the ECLS-K as well, yielding an additional 8,861 
possible new private schools. Since the number of kindergartners enrolled in these schools was unknown, 
a minimum kindergarten enrollment was assumed for sampling purposes (typically 24, unless the name 
was suggestive of day care, in which case 12 was assumed). 

The supplemental frame contained 11,405 private schools. A sample of 279 schools was 
selected, using systematic sampling with a probability proportional to these imputed enrollments. Each 
sampled school was contacted by telephone and screened to ascertain whether the school was public or 
private, whether it would be open in academic year 1998-1999; and whether it would offer kindergarten. 
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If the school met all of these conditions and was not Catholic, the school was eligible and released for 
data collection. 

A second supplemental procedure involved contacting local education agencies (LEAs) and 
local government offices for information on non-Catholic private schools. This procedure was done only 
in the smallest ECLS-K PSUs, on the theory that if these PSUs had coverage problems their large weights 
were likely to introduce a larger bias in the estimates. All LEAs within these PSUs were contacted by 
telephone. For each city/town within the PSU, a list of local government offices was compiled using the 
blue pages. Successive government offices were called within a city or town until one was found that 
could provide information on private schools. As with the yellow pages, new schools identified by LEAs 
and local government offices were unduplicated against the PSS file before being added to the new school 
frame. Since kindergarten enrollment was unknown, it was imputed as described in the previous 
paragraph and sampling was performed using systematic sampling with probability proportional to size. 
The LEA search resulted in the identification of 30 new private schools after unduplication, of which 14 
were sampled. The local government search yielded 19 new schools, of which 8 were sampled. Finally, 
three additional new private schools were reported by field staff based on personal knowledge. Of these, 
two schools were sampled. The same screening procedures to ascertain whether the school was public or 
private; whether it would be open in academic year 1998-1999; and whether it would offer kindergarten 
were then applied to these sampled schools. 

The total number of non-Catholic private schools that were sampled was 303. After the 
screening procedures were applied, only 109 of these schools were eligible. These 109 schools are 
referred to as the supplemental sample of non-Catholic private schools. 

The final ECLS-K school sample for the base year was 1,413 schools, including 953 public 
schools, 123 Catholic schools, and 337 non-Catholic private schools. Of these, 136 schools (72 percent 
private) were later found to be ineligible because they did not have any kindergarten programs; three 
schools participated in fall-kindergarten, but not in spring-kindergarten (1 public and 2 private); 259 
schools (38 percent private) refused to participate in both fall and spring; and 65 schools (42 percent 
private) refused to participate in the fall but were converted to cooperating schools in the spring during 
the spring refusal conversion. At the end of the base year, 1,014 schools were still participating in the 
ECLS-K. 
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3.2.3 



Sampling Children, Parents, and Teachers Within Schools 



The goal of the student sample design was to obtain an approximately self-weighting sample 
of students to the extent possible while achieving the minimum required sample size for APIs (the only 
subgroup that needed to be oversampled to meet the study’s precision goals). Two independent sampling 
strata were formed within each school, one containing API students and the second all other students. 
Within each stratum, students were selected using equal probability systematic sampling, using a higher 
rate for the API stratum. In general, the target number of children sampled at any one school was 24. The 
actual sample size per school ranged from 1 to 28. If one twin was selected into the sample then both 
twins were included, raising the maximum number of children to sample from 24 to 28 in a small number 
of schools. Once the sampled children were identified, parent contact information was obtained from the 
school and was used to identify a parent or guardian for the parent interview. 

During the fall-kindergarten data collection, a census of kindergarten teachers was taken at 
each school. In spring-kindergarten, new teachers who had joined the schools and teachers in schools 
participating after the fall were added to the census of teachers. In the spring-first and spring-third grade 
data collections, the only teachers included were the teachers of the sampled children. For every data 
collection, each sampled child was li nk ed to his or her teacher. A child could be linked to only one 
general education teacher. In cases where a child had more than one general education teacher, a 
‘primary’ teacher was identified for the child. In addition, special education teachers and service 
providers were li nk ed to sample cases who received such services. As with the general education 
teachers, a child would be li nk ed to only one special education teacher or service provider. Details on the 
linking of teachers to the children are found in chapter 4. 



3.3 Fall-First Grade Subsample 

The fall data collection consisted of a 30 percent sample of schools containing 
approximately 25 percent of the base year students eligible to participate in the second year. The goal of 
this subsample was to measure the extent of summer learning loss and the factors that contribute to such 
loss and to better disentangle school and home effects on children’s learning 
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3.3.1 



PSU Sample 



A subsample of ECLS-K PSUs was selected for the fall-first grade data collection. All 24 of 
the self-representing PSUs were retained. Of the 76 non-self-representing PSUs, 38 were retained by 
sampling one PSU per stratum with equal probability. 



3.3.2 School Sample 

Base year schools in the 62 fall-first grade sampled PSUs were stratified by frame source 
(original public, original private, supplemental public, and supplemental private as described in 
section 3.2. 2. 8) and arranged in their original selection order. A 30 percent equal probability sample of 
schools was drawn in the 24 self-representing PSUs and a 60 percent sample of schools was drawn in the 
38 non-self-representing PSUs. In total 311 schools that had cooperated in either fall- or spring- 
kindergarten were selected. The characteristics of the base year cooperating schools selected for fall-first 
grade are presented in table 3-10. 



3.3.3 Child Sample 

Fall-first grade data collection consisted of the direct child assessment and the parent 
interview. Data collection was attempted for every eligible child found still attending the school in which 
he or she had been sampled during kindergarten. “Eligible” was defined as a base year respondent (i.e., a 
child who had either a fall- or spring-kindergarten child assessment or parent interview or was excluded 
from assessment because of a disability or because the child belonged in the language minority (not 
Spanish) group. Base year nonrespondents were not sampled and were handled by adjusting the weights 
(see section 7. 2. 1.2.1 for details of adjustment for base year nonresponse). 

Because of the additional burden of school recruiting, the cost of collecting data for a child 
who had transferred from the school in which he or she was originally sampled exceeds that for a child 
who stayed enrolled. To contain these costs, a random 50 percent of children were subsampled to be 
followed for fall-first grade data collection in the event that they had transferred. 
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Table 3-10. Characteristics of base year cooperating schools selected for fall-first grade: School year 
1999-2000 



Characteristic 


Total 


Public 


Private 


Total 


311 


228 


83 


Region 


Northeast 


57 


39 


18 


Midwest 


83 


59 


24 


South 


99 


77 


22 


West 


72 


53 


19 


Type of locale 


Large city 


62 


42 


20 


Midsize city 


59 


45 


14 


Urban fringe of large city 


86 


61 


25 


Urban fringe of midsize city 


18 


14 


4 


Large town 


15 


12 


3 


Small town 


28 


19 


9 


Rural 


43 


35 


8 


School affiliation 


Public 


228 


228 


t 


Catholic 


29 


t 


29 


Other religious 


33 


t 


33 


Nonreligious, private 


21 


t 


21 


School type 


Regular 


292 


222 


70 


Ungraded 


1 


1 


0 


No grade beyond kindergarten 


18 


5 


13 



t Not applicable. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), fall 1999 and spring 2000. 



Except for children who were repeating kindergarten, all base year children sampled in 
schools with a high grade of kindergarten are de facto movers. Since many of these movers might move 
en masse to the same first-grade school, steps were taken to follow these children at a higher rate. Using 
the information collected during spring-kindergarten, a list of destination schools was compiled for each 
such school. The destination school having the most movers was designated as primary, unless no such 
school had more than three movers. Children who moved en masse into a primary destination school in 
fall-first grade were treated as “nonmovers” and were not subsampled, that is, they continued to be 
followed and were part of the ECLS-K sample. All other movers were sampled at the rate of 50 percent. 
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As discussed above, a random 50 percent of children were subsampled to be followed if they 
moved out of the kindergarten school. Sampling was done with equal probability. Prior to sampling, 
children were stratified into groups of nonmovers, movers with information identifying their new schools, 
and movers without such identifying information. A flag was created for each child indicating whether the 
child had been sampled to be followed. 



3.4 Spring-First Grade Sample 

The ECLS-K spring- first grade data collection targeted all base year respondents and not just 
the fall- first grade subsample. Hence, the sample includes children who were assessed and whose parents 
were interviewed in fall- or spring-kindergarten, as well as the 70 children who could not be assessed in 
fall- or spring-kindergarten because of a disability or because they belonged in the language minority (not 
Spanish) group. In addition, the spring student sample was freshened to include current first-graders who 
had not been enrolled in kindergarten in 1998-99 and therefore had no chance of being included in the 
ECLS-K base year kindergarten sample. This group includes children who skipped kindergarten 
altogether in 1998-99, children who attended a kindergarten program outside of the U.S. in 1998-99, and 
children who were in first grade in 1998-99 and repeating it in 1999-2000. While all students still 
enrolled in their base year schools were recontacted, only a 50 percent subsample of base year sampled 
students who had transferred from their kindergarten school was followed for data collection. 



3.4.1 Subsampling Movers 

In spring-first grade all children in a random 50 percent subsample of base year schools were 
flagged to be followed for data collection if they transferred from their base year school. (This is in 
contrast to fall-first grade where a random 50 percent of children in each of the 30 percent of schools 
subsampled were flagged.) In order to maximize the amount of longitudinal data, care was taken during 
spring-first grade sampling to ensure that any child who had been flagged to be followed in fall- first grade 
would continue to be followed. 

In selecting the spring-first grade 50 percent subsample of schools where movers would be 
flagged for followup, the three primary strata were self-representing PSUs, non-self-representing PSUs 
that had been selected for fall-first grade, and non-self-representing PSUs that had not been selected for 
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fall-first grade. Within these major strata, schools were grouped by frame source (original public, original 
private, supplemental public, and supplemental private). Finally within each frame source, schools were 
stratified by whether the school participated in the base year study, and arranged in original selection 
order. Schools that had been part of the 30 percent fall-first grade sample were automatically retained. 
Then equal probability sampling methods were employed to augment the sample to the desired 50 percent 
of schools. The net result of these procedures was that every base year selected school had a 50 percent 
chance of having its ECLS-K movers followed during spring-first grade, and any mover who had been 
followed in fall-first grade would still be followed in spring- first grade. 



3.4.2 Sample Freshening 

As noted earlier, a sample freshening procedure was used to make it possible to produce 
estimates of all children enrolled in first grade in the spring of 2000. The spring-first grade student 
freshening used a half-open interval sampling procedure (Kish 1965). The procedure was implemented in 
the same 50 percent subsample of ECLS-K base year schools where movers were flagged for followup. 
Each of these schools was asked to prepare an alphabetic roster of students enrolled in first grade and the 
names of ECLS-K kindergarten-sampled children were identified on this list. Beginning with the name of 
the ECLS-K first kindergarten-sampled child, school records were checked to see whether the student 
directly below in the sorted list attended kindergarten in the United States in fall 1998. If not, (1) that 
child was considered to be part of the freshened sample and was linked to the base year sampled student 
(i.e., was assigned that student’s probability of selection), and (2) the record search procedure was 
repeated for the next listed child, and so forth. When the record search revealed that a child had been 
enrolled in kindergarten the previous year, that child was not considered part of the freshened sample and 
the procedure was resumed with the second base year ECLS-K sampled student name, and so on. 8 
Student freshening brought 165 first graders into the ECLS-K sample, which increased the weighted 
survey estimate of the number of first graders in the United States by about 2.6 percent. 

The student freshening procedure was not entirely free of bias. A first grader would have no 
chance of being in the ECLS-K first grade sample if he or she was enrolled in a school where neither the 
child nor any of his or her classmates had attended kindergarten in the United States in fall 1998. This 
would be a rare circumstance and is not thought to be an important source of bias. A more significant 



8 The student roster was “circularized” (i.e., the first name on the roster was considered to follow the last name on the roster in the 
implementation of the procedure). 
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source of potential bias is nonresponse. One source of nonresponse inherent to the freshening plan was 
that the procedure only involved students who had not transferred from the school in which they had been 
sampled during the base year. Another source of nonresponse that also affected the freshening procedure 
were schools that refused to provide or could not provide the necessary information, such as alphabetic 
roster of students enrolled in first grade or whether students had attended kindergarten the previous year. 
The school freshening completion rate is slightly higher for public schools than for private schools. Of the 
494 schools eligible for freshening, 380 are public schools and 114 are private schools. Ninety four 
percent of the public schools and 93 percent of the private schools participated in the freshening process. 



3.5 Spring-Third Grade Sample 

The procedures used in spring-first grade to subsample movers reduced the loss in sample 
size and reduced data collection costs since movers cost considerably more to interview than nonmovers. 
These procedures were also used for the ECLS-K third-grade data collection with some modifications. 
One reason for modifying the procedures was that some children had already moved out of their original 
school, and some of the movers were sampled and some were not. In addition, there were concerns about 
special domains of interest and methods that might be used to increase the sample size for the children in 
these groups. Results from the first-grade collection were used to address these third-grade sample design 
issues. 



3.5.1 Estimates from Spring-First Grade 

Table 3-11 presents data on the outcome of the spring- first grade data collection activities, 
by subgroups of interest and by mover status. In this table and subsequent tables, school affiliation and 
type of locale are those of the original sample schools. Race/ethnicity and language characteristics of the 
children are from the ECLS-K base year data, which are available for all children. Data from first grade 
are only available for first-grade respondents. For children sampled in first grade, data are from spring- 
first grade. The table shows that overall 26 percent (5,477) of the children moved from the school they 
were sampled in, about 48 percent (2,620) of these movers were sampled, and the unweighted completion 
rate for movers was 83 percent (1,967 mover respondents). For nonmovers, the completion rate was 
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Table 3-11. Spring-first grade data collection results by mover status: School year 1999-2000 



All sampled children 


Mover status 


Movers 


Nonmovers 












Unweighted 


Movers 














Unweighted 








Unweighted 


Subgroup 1 






Not 




completion 


not 


Total 


Percent 




Not 




Percent 


completion 




Not 




completion 


Total 2 


Respond 3 


respond 4 


Ineligible 


rate 5 


sampled 


movers 


moved 


Respond 3 


respond 4 


Ineligible 


sampled 


rate 5 


Respond 3 


respond 4 


Ineligible 


rate 5 


All 


21,331 


17,324 


899 


251 


95.1 


2,857 


5,477 


25.7 


1,967 


403 


250 


47.8 


83.0 


15,357 


496 


i 


96.9 


School affiliation 




































Public 


16,761 


13,661 


710 


221 


95.1 


2,169 


4,189 


25.0 


1,466 


334 


220 


48.2 


81.4 


12,195 


376 


i 


97.0 


Private 


4,570 


3,663 


189 


30 


95.1 


688 


1,288 


28.2 


501 


69 


30 


46.6 


87.9 


3,162 


120 


0 


96.3 


Catholic 


2,354 


2,031 


66 


12 


96.9 


245 


433 


18.4 


150 


26 


12 


43.4 


85.2 


1,881 


40 


0 


97.9 


Non-Catholic 


2,216 


1,632 


123 


18 


93.0 


443 


855 


38.6 


351 


43 


18 


48.2 


89.1 


1,281 


80 


0 


94.1 


Type of locale 




































Rural 


2,509 


2,227 


86 


14 


96.3 


182 


428 


17.1 


194 


39 


13 


57.5 


83.3 


2,033 


47 


1 


97.7 


Non-Rural 


18,822 


15,097 


813 


237 


94.9 


2,675 


5,049 


26.8 


1,773 


364 


237 


47.0 


83.0 


13,324 


449 


0 


96.7 


Race/ethnicity l 6 




































Hispanic 


3,777 


2,988 


164 


81 


94.8 


544 


1,048 


27.7 


349 


75 


80 


48.1 


82.3 


2,639 


89 


1 


96.7 


Black 


3,229 


2,468 


150 


69 


94.3 


542 


1,066 


33.0 


374 


81 


69 


49.2 


82.2 


2,094 


69 


0 


96.8 


Asian/Pacific 




































Islander 


1,579 


1,291 


61 


18 


95.5 


209 


359 


22.7 


114 


18 


18 


41.8 


86.4 


1,177 


43 


0 


96.5 


Other 


12,746 


10,577 


524 


83 


95.3 


1,562 


3,004 


23.6 


1,130 


229 


83 


48.0 


83.1 


9,447 


295 


0 


97.0 


Race/ethnicity 2 6 




































Hispanic 


3,698 


2,926 


160 


81 


94.8 


531 


1,023 


27.7 


340 


72 


80 


48.1 


82.5 


2,586 


88 


1 


96.7 


Black 


3,229 


2,468 


150 


69 


94.3 


542 


1,066 


33.0 


374 


81 


69 


49.2 


82.2 


2,094 


69 


0 


96.8 


Asian/Pacific 




































Islander 


1,867 


1,537 


69 


20 


95.7 


241 


429 


23.0 


144 


24 


20 


43.8 


85.7 


1,393 


45 


0 


96.9 


Other 


12,537 


10,393 


520 


81 


95.2 


1,543 


2,959 


23.6 


1,109 


226 


81 


47.9 


83.1 


9,284 


294 


0 


96.9 


Language minority 




































Non-English 


5,372 


4,317 


228 


107 


95.0 


720 


1,397 


26.0 


472 


98 


107 


48.5 


82.8 


3,845 


130 


0 


96.7 


English 


15,959 


13,007 


671 


144 


95.1 


2,137 


4,080 


25.6 


1,495 


305 


143 


47.6 


83.1 


11,512 


366 


1 


96.9 



1 Characteristics of the schools (school affiliation and type of locale) are from the original sample schools. 

2 The total number of children excludes 68 children who responded in fall-kindergarten and became ineligible in spring-kindergarten, and includes 139 children sampled in first grade who responded. 

3 A respondent is a child with assessment data or parent interview data, or a child who could not be assessed due to a disability. 

4 Nonrespondents include those who did not participate fully and movers who could not be located. 

5 The unweighted completion rate was computed as the number of respondents divided by the sum of respondents and nonrespondents. 

6 Race/ethnicity 1 was the strict definition of API (RACE=5-Asian, or 6-Native Hawaiian or other Pacific Islander), while race/ethnicity 2 was the broader definition (RACE=5 -Asian, or 6-Native 
Hawaiian or other Pacific Islander or WKASIAN=1 -Child is Asian, or WKPACISL=1 -Child is Native Hawaiian or other Pacific Islander). Variables are from the ECLS-K base year data file. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 kindergarten (ECLS-K), fall 1999 and spring 
2000 . 



97 percent (15,357 nonmover respondents). A child was considered a respondent in these computations if 
either the child assessment or the parent interview was completed for spring-first grade or the child was 
not assessed due to a disability. The completion rate in this table was computed as the number of 
respondents divided by the sum of respondents and nonrespondents. Nonrespondents include those who 
did not participate fully and movers who could not be located. For first grade, 269 of the movers who 
were sampled for follow-up could not be located, or about 1 1 percent of all movers eligible for the first 
grade data collection. 

The mover rates show the types of variation that had been expected, with higher mover rates 
for Black and Flispanic children, for example. A total of 39 percent of the children in non-Catholic private 
schools moved to other schools when they advanced from kindergarten to first grade (855 movers as 
shown in table 3.1 1). Seventy-six percent of children who moved from kindergarten in private schools to 
first grade in public schools attended non-Catholic private schools in kindergarten, about three times the 
number of children who moved from Catholic to public schools. 

One of the concerns in using the kindergarten to first-grade mover rates to make estimates 
for future transitions was whether the mover rates for the 1-year time period between kindergarten and 
first grade were reasonable when applied to the transition between first and third grade. One might argue 
that a 2-year period should result in a higher mover rate than the 1 -year rate. Flowever, parents may be 
more reluctant to change the school for a child between first and third grade than between kindergarten 
and first grade. Kindergarten is also special for other reasons. For example, the availability of full- and 
part-day classes may be an important factor in the choice of the kindergarten. There are no other data 
sources that could be used to examine differential mover rates between years. As a result, the 1-year 
moving rates in table 3-11 were applied to the 2-year period between first and third grade after adding 
another 5 percent to the rates to account for the 2-year period. An exception was made for children who 
attended non-Catholic private schools in the base year and had the highest rates of moving among all the 
domains examined. This was assumed to be a special case for kindergarten and the average mover 
percentage was applied to these children for the third grade. 

The other main concern was whether the extremely high completion rate for nonmovers 
(97 percent) could be duplicated in future years. To be more conservative and to account for the fact that 
nonrespondents from earlier rounds (i.e., base year respondents were included in the third-grade sample 
whether or not they responded in first grade) were included in subsequent rounds of data collection, it was 
assumed that a 95 percent completion rate would be achieved for nonmovers in third grade. 
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3.5.2 



Third-Grade Sample Design 



The basic plan for third grade was the plan implemented for first grade where only 50 
percent of the children who moved from the original sample schools were followed into their new 
schools. This plan was modified for third grade as described below. To be eligible for the third-grade 
sample, a child had to have been a base year respondent or sampled in first grade. Children who moved 
out of the country or died were excluded (i.e., ineligible). The following children were fielded for third 
grade: 



■ All the children responding in the base year who remained in their original schools, 
where the original schools also included destination schools (described later). 

■ All the children who moved from an original school in a previous wave of data 
collection and were retained in the subsample of movers for that wave. For example, 
if a child moved between kindergarten and first grade and was part of the 50 percent 
subsample that was followed, then the child would be retained for future rounds 
without subsampling as long as the child remained eligible. 

■ A subsample of 50 percent of the children who moved from their original school at 
any time after the base year. For example, a child who moved between first grade and 
third grade would be subject to subsampling and had a 50 percent chance of being 
included in the third-grade followup. In alternatives discussed later, differential 
subsampling rates were introduced. 

To prevent an accumulation of nonresponse, the ECLS-K design does not use the approach 
of many longitudinal studies that exclude sampled units from future rounds if they did not respond in a 
particular wave. Instead, the basic plan was modified so that all eligible base-year respondents who were 
sampled in the first-grade followup would be eligible for the third-grade followup even if they did not 
respond in the first grade. Even though the participation rate for first-grade nonrespondents might be 
lower compared with first-grade respondents in the subsequent followups, the effort was an attempt to 
increase overall response rates by including first-grade nonrespondents in third grade. The approach is 
also consistent with the analytic use of the data for the ECLS-K, since many analyses may include less 
than complete wave responses. For example, a change in scores from kindergarten to third grade for 
subgroups is an important analytic objective, and it can be estimated without complete data at each wave. 



A second procedure that was part of the modification of the basic plan for the third-grade 
followup was an extension of a procedure that was used in the first-grade followup to deal with schools 
that ended with kindergarten (i.e., kindergarten was the highest grade offered). A school was called a 
destination school if at least 4 students from a school ending in kindergarten attended this school in first 
grade. For the third grade 28 original schools ended in second grade, and 3 of the destination schools 
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identified in first grade ended in second grade. In total, 3 percent of all eligible first graders in the 
ECLS-K sample attended schools ending in second grade. As was done for the first-grade sample, 
children in the destination schools were treated as nonmovers for the third-grade sample. As nonmovers, 
they were all followed into their new schools, resulting in a 2 percent increase of the third-grade sample 
size over that which would result if 50 percent of these children were subsampled out as movers. 



3.5.3 Expected Sample Size 

Table 3-12 gives the expected sample sizes of children in third grade by subgroups of 
interest and mover status for the basic plan. In this table, a respondent is defined as a child with either a 
complete child assessment or parent interview, or a child who could not be assessed because of a 
disability. This table shows that the expected number of children with completed child assessments or 
parent interviews in the third grade was 14,304 under the assumed mover rates (differential by subgroups 
as shown in the table), subsampling rate (47 percent instead of 50 percent to account for ineligibility in 
third grade), and completion rate (95 percent). The estimates for the selected groups of high interest are 
given in the rows below. The third from last column is the estimated design effect resulting from 
sampling movers and nonmovers differentially. It does not include any other factors such as clustering. 
The next to last column is an estimate of the number of new schools that would enter the ECLS-K 
sample. The last column is an estimate of the total number of schools that sampled children would be 
attending, assuming 1.5 movers attended the same school on average. 
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Table 3-12. Expected sample size for selected subgroups for third grade, by mover status: School year 2001-02 



Sampled from first grade 


Mover 2 


Completion rate 2 


Sample of 
new movers 


Total responding (expected) 


Subsample 

design 

effect 4 


New third 
grade 
schools 5 


Total 

schools 


Subgroup 1 


Total 


Movers 


Nonmovers 


Rate 


Sampled 3 


Movers Nonmovers 


Total 


Movers 


Nonmovers 


All children 


18,223 


2,370 


15,853 


— 


— 


— 


— 


2,284 


14,304 


3,860 


10,444 


t 


1,523 


4,144 6 


School affiliation 






























Public 


14,371 


1,800 


12,571 


30 


47 


81 


95 


1,772 


11,270 


2,909 


8,361 


1.17 


1,182 


— 


Private 


3,852 


570 


3,282 


33 


47 


88 


95 


512 


3,034 


951 


2,083 


1.14 


341 


— 


Catholic 


2,097 


176 


1,921 


23 


47 


85 


95 


211 


1,728 


330 


1,398 


1.13 


— 


— 


Non-Catholic 


1,755 


394 


1,361 


31 


47 


89 


95 


198 


1,420 


528 


892 


1.27 


— 


— 


Type of locale 






























Rural 


2,313 


233 


2,080 


22 


47 


83 


95 


216 


1,914 


374 


1,540 


1.09 


— 


— 


Non-rural 


15,910 


2,137 


13,773 


32 


47 


83 


95 


2,080 


12,402 


3,482 


8,920 


1.17 


— 


— 


Race/ethnicity l 5 






























Hispanic 


3,152 


424 


2,728 


33 


47 


82 


95 


420 


2,438 


695 


1,743 


1.18 


t 


t 


Black 


2,618 


455 


2,163 


38 


47 


82 


95 


386 


1,965 


692 


1,274 


1.17 


t 


t 


Asian/Pacific Islander 


1,352 


132 


1,220 


28 


47 


86 


95 


159 


1,089 


251 


838 


1.17 


t 


t 


Other 


11,101 


1,359 


9,742 


29 


47 


83 


95 


1,308 


8,829 


2,218 


6,611 


1.14 


t 


t 



See notes at end of table. 
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Table 3-12. Expected sample size for selected subgroups for third grade, by mover status: School year 2001-02 — Continued 



Sampled from first grade 


Mover 2 


Completion rate 2 


Sample of 
new movers 


Total responding (expected) 


Subsample 


New third 


Total 

schools 


Subgroup 1 


Total 


Movers 


Nonmovers 


Rate 


Sampled 3 


Movers Nonmovers 


Total 


Movers 


Nonmovers 


design 

effect 4 


grade 

schools 5 


Race/ethnicity 2 5 
Hispanic 


3,086 


412 


2,674 


33 


47 


83 


95 


411 


2,389 


679 


1,711 


1.18 


t 


t 


Black 


2,618 


455 


2,163 


38 


47 


82 


95 


386 


1,965 


692 


1,274 


1.17 


t 


t 


Asian/Pacific Islander 


1,606 


168 


1,438 


28 


47 


86 


95 


189 


1,290 


306 


984 


1.16 


t 


t 


Other 


10,913 


1,335 


9,578 


29 


47 


83 


95 


1,288 


8,675 


2,179 


6,497 


1.15 


t 


t 


Language minority 
Non-English 


4,545 


570 


3,975 


31 


47 


83 


95 


579 


3,557 


952 


2,605 


1.17 


t 


t 


English 


13,678 


1,800 


11,878 


31 


47 


83 


95 


1,706 


10,747 


2,912 


7,835 


1.15 


t 


t 



— Not available, 
f Not applicable. 

1 Characteristics of the schools (school affiliation and type of locale) are from the original sample schools. 

2 The mover rates and the completion rates for movers are those used in the computation of expected sample size, and are differential by subgroups. Since no “total” rates were used in the computation, 
they are not available. 

3 The sampling rate for movers is set at 47 percent (instead of 50 percent) to account for ineligibility of students in future rounds. 

4 The design effects in this column are the results of sampling movers and nonmovers differentially. They do not include the effect of clustering. 

5 The number of new third-grade schools is estimated as 1.5 schools per sampled new mover. 

6 2,621 schools from kindergarten/first grade plus the new third-grade schools. 

7 Race/ethnicity 1 was the strict definition of API (RACE=5 -Asian or 6-Native Hawaiian or other Pacific Islander), while race/ethnicity 2 was the broader definition (RACE=5 -Asian or 6-Native 
Hawaiian or other Pacific Islander or WKASIAN=1 -Child is Asian or WKPACISL=1 -Child is Native Hawaiian or other Pacific Islander). Variables are from the ECLS-K base year data file. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 kindergarten (ECLS-K), spring 2002. 



3.5.4 



Protecting the Language Minority Children 



Special attention was paid to language minority and API children to ensure that the sample 
sizes would be large enough to support analytic goals in developing the sampling plans for the third 
grade. Children in the language minority group are children whose home language is non-English or who 
were screened using the Oral Language Development Scale (OLDS) prior to assessments during the base 
year (or first grade for freshened children). 9 Two classifications of APIs are shown in tables 3-11 and 3- 
12. The first classification was identified using a strict definition of API, i.e., if the child was identified 
only as API by the composite race variable (RACE = 5-Asian or 6-Native Hawaiian or other Pacific 
Islander). The second classification was identified using a broader definition, i.e., if a child was identified 
only as API as in the strict definition (RACE = 5-Asian or 6-Native Hawaiian or other Pacific Islander) or 
if a child has positive answers to the API race identification variables (WKASIAN = 1 -Child is Asian or 
WKPACISL = 1 -Child is Native Hawaiian or other Pacific Islander). The variables RACE, WKASIAN 
and WKPACISL are in the base year data file. The broader definition of API yields a larger population of 
children. 



After reviewing the expected yields without oversampling, it was decided to increase only 
the sample size for children belonging to the language minority group. Beginning in third grade, these 
children would not be subsampled for followup if they moved from their original school. Instead, data 
collection would be attempted for all language minority children. Table 3-13 is analogous to table 3-12 
but is adjusted for this approach of retaining all movers in the language minority group (in practice the 
subsampling rates are shown as 95 percent because some children became ineligible). 

One consequence of protecting this subgroup is to increase the sample size and precision for 
the subgroup. The design effect due to subsampling is slightly lower under this plan because a smaller 
proportion of the movers were subsampled than under the basic plan (only the movers that were already 
subsampled in first grade are subsampled). Another consequence is that the number of schools that the 
sampled children attended increased. Because all language minority children were followed, table 3-13 
shows an expected increase of 395 schools in third grade (1,918 - 1,523 = 395). 



9 Information about home language came from the parent interview and whether or not children were screened with the OLDS was based on 
information provided by their schools (see the Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K) Base Year Public- 
Use Data Files and Electronic Code Book: User’s Manual (NCES 2001-029r) (Tourangeau, Burke, et al. 2004). 
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Table 3-13. Expected sample size for third grade, by mover status, if standard subsampling rates were used and if language minority children 
were preserved: School year 2001-02 





Sampled from first grade 


Mover 1 




Completion rate 1 




Sample of 
new movers 


Total responding (expected) 


Subsample 


New third 


Total 

schools 5 


Sampling approach 


Total 


Movers 


Nonmovers 


Rate 


Sampled 2 


Movers Nonmovers 


Total 


Movers 


Nonmovers 


design 

effect 3 


grade 

schools 4 


Using standard 
































subsampling rates 


18,223 


2,370 


15,853 


— 




— 


— 


— 


2,285 


3,864 


10,440 


14,304 


t 


1,523 


4,144 


Non-English 


4,545 


570 


3,975 


31 




47 


83 


95 


579 


952 


2,605 


3,557 


1.17 


t 


t 


English 


13,678 


1,800 


11,878 


31 




47 


83 


95 


1,706 


2,912 


7,835 


10,747 


1.15 


t 


t 


Preserving the language 


minority group 


18,223 


2,370 


15,853 


— 




— 


— 


— 


2,877 


4,354 


10,440 


14,794 


t 


1,918 


4,539 


Non-English 


4,545 


570 


3,975 


31 




95 


83 


95 


1,171 


1,442 


2,605 


4,047 


1.04 


t 


t 


English 


13,678 


1,800 


11,878 


31 




47 


83 


95 


1,706 


2,912 


7,835 


10,747 


1.15 


t 


f 



— Not available, 
f Not applicable. 

1 The mover rates and the completion rates for movers are those used in the computation of expected sample size, and are differential by subgroups. Since no “total” rates were used in the computation, 
they are not available. 

2 The sampling rate for movers is set at 47 percent (instead of 50 percent) to account for ineligibility of students in future rounds. If the language minority group is preserved, it is set at 95 percent. 

3 The design effects in this column are the results of sampling movers and nonmovers differentially. They do not include the effect of clustering. 

4 The number of new third-grade schools is estimated as 1.5 schools per sampled new mover. 

5 2,621 schools from kindergarten/first grade plus the new third-grade schools. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2002. 



3.5.5 



Precision Requirements 



When the precision estimates were computed from the kindergarten sample at the end of the 
base year, higher than expected design effects for assessment scores were observed. The design effects for 
most other statistics, such as proportions of children with a particular characteristic, were moderate and 
within the range expected (1.6 to 6.9 for proportions greater than 30 percent for an average of 4.0). The 
design effects for assessment scores (4.5 to 9.5 for an average of 6.9) were investigated and found to be 
correct and unrelated to data collection artifacts. For example, interviewer effects were found to be 
negligible and did not bias assessment scores. The design effects for test scores were much larger than the 
average of 3.8 that was expected at the design stage. For all students, the design effects for math and 
reading scores were about 6.5, while for general knowledge the design effects were even larger, at 7.7. 
For design effects from the base year, see chapter 4 of the ECLS-K Base Year Public-Use Data Files and 
Electronic Code Book: User’s Manual (NCES 2001-029r) (Tourangeau, Burke, et al. 2004). These larger 
design effects are one component that affects the ability of the survey to meet the precision requirements 
as described in section 3.1. 

The spring-third grade estimates of design effects are similar to those in the earlier rounds 
and are larger than had been predicted prior to any data collection. The longitudinal estimates have design 
effects that are not as large as might be expected given the larger cross-sectional design effects. In fact, 
the correlations for mean test scores are as high as 0.8 to 0.9. The higher than expected correlations make 
estimates of changes in scores over time more precise, thus it is possible to meet the precision 
requirements for estimates of change with smaller sample sizes. Table 3-14 shows that the sample sizes 
for the key analytic subgroups (public, Catholic, non-Catholic, Hispanic, Black, Asian/Pacific Islander, 
other races together, and language minority) were expected to be at least 1,000. 10 Samples of this size 
were expected to be sufficient for estimating most characteristics. For example, test scores were expected 
to have a coefficient of variation of about 3 percent with samples of 1,000. More details on estimates of 
design effects can be found in chapter 4 of the ECLS-K User’s Manual for the First Grade Public-Use 
Data Files and Electronic Code Book (NCES 2002-135) (Tourangeau et al. 2002) and the ECLS-K User’s 
Manual for the Third Grade Public-Use Data File and Electronic Code Book (NCES 2004-001) 
(Tourangeau, Brick, et al. 2004). 



10 The sample sizes for key analytic groups are the totals for each group, not for each cell in table 3-14. 



3-38 




Table 3-14. Characteristics of third-grade respondents — number of third-graders by subgroup: 
School year 2001-02 



Expected 2 


Achieved 


Subgroup 1 


Total 


Language 

minority 


Not 

language 

minority 


Total 


Language 

minority 


Not 

language 

minority 


Total 


14,794 


4,047 


10,747 


15,305 


4,041 


11,264 


School affiliation 














Public 


11,643 


3,356 


8,287 


12,070 


3,374 


8,696 


Private 


3,151 


691 


2,460 


3,235 


667 


2,568 


Catholic 


1,625 


373 


1,252 


1,817 


383 


1,434 


Non-Catholic 


1,526 


318 


1,208 


1,418 


284 


1,134 


Type of locale 














Rural 


1,712 


210 


1,502 


2,005 


222 


1,783 


Non-rural 


13,082 


3,837 


9,245 


13,300 


3,819 


9,481 


Race/ethnicity l 3 














Hispanic 


2,773 


2,165 


608 


2,752 


2,156 


596 


Black 


2,188 


128 


2,060 


2,007 


118 


1,889 


Asian/Pacific Islander 


1,165 


962 


203 


1,174 


947 


227 


Other 


8,668 


792 


7,876 


9,372 


820 


8,552 


Race/ethnicity 2 3 














Hispanic 


2,716 


2,130 


587 


2,691 


2,116 


575 


Black 


2,188 


128 


2,060 


2,007 


118 


1,889 


Asian/Pacific Islander 


1,369 


1,057 


312 


1,404 


1,056 


348 


Other 


8,520 


732 


7,788 


9,203 


751 


8,452 



1 Characteristics of the schools (school affiliation and type of locale) are from the original sample schools. 



2 The expected sample size was computed using assumed mover rates (differential by subgroups as shown table 3-12), a 47 percent subsampling 
rate (instead of 50 percent to account for ineligibility in third grade), and a 95 percent completion rate. 

3 Race/ethnicity 1 was the strict definition of API (RACE=5 -Asian, or 6-Native Hawaiian or other Pacific Islander), while race/ethnicity 2 was the 
broader definition (RACE=5 -Asian, or 6-Native Hawaiian or other Pacific Islander or WKASIAN=1 -Child is Asian, or WKPACISL=1 -Child is 
Native Hawaiian or other Pacific Islander). Variables are from the ECLS-K base year data file. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2002. 



3.5.6 Spring-Third Grade Sampling Outcome 



To summarize, the sample of children for spring-third grade consists of all children who 
were base year respondents and children who were brought into the sample in spring-first grade through 
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the sample freshening procedure. Sample freshening was not implemented in third grade; hence no new 
students entered the sample. 

While all students still enrolled in their base year schools were recontacted, slightly more 
than 50 percent of the base year sampled students who had transferred from their kindergarten school 
were followed for data collection. This subsample of students was the same 50 percent subsample of base 
year movers followed in spring-first grade, including the movers whose home language was not English 
(language minority students). Children who were followed in spring-first grade were retained in the 
sample (i.e., the mover followup still targeted the same 50 percent subsample of children in the base year 
schools). In addition, children whose home language was not English and who had moved between 
spring-first grade and spring-third grade were all retained rather than being subsampled at the 50 percent 
rate. If they had moved before first grade, they were not to be followed. This modification to the mover 
followup procedure provided a larger sample of children whose home language is not English for analytic 
purposes. The mover followup activities that originally targeted a 50 percent subsample of children in 
base year schools resulted in a 54 percent subsample with the addition of language minority children. 

Tables 3-14 (count) and 3-15 (percent) show the characteristics of the achieved third-grade 
sample compared with the expected third-grade sample. The total number of children in the language 
minority group is virtually the same as the expected number while the total number of children in the 
other group is about 5 percent larger than the expected number. In computing the expected sample size, 
the same mover rate was assumed for both groups of children. The third-grade sample shows that the non- 
language minority children moved at a lower rate (42 percent) than the language minority children (44 
percent) resulting in a slightly larger sample of non-language minority children. The agreement between 
the expected and achieved sample sizes is rather remarkable given the numerous assumptions required. 
The actual percent distribution of third-graders within each subgroup is as expected with the exception of 
the Catholic and non-Catholic private schools where the percent of children in Catholic schools is higher 
than that of children in non-Catholic private schools. This may be due to the lower completion rate of 
children in non-Catholic private schools compared with children in Catholic private schools (93 percent 
and 97 percent, respectively). Elsewhere among the children in the language minority group, the 
difference between the expected distribution and the actual distribution is less than 1 percent. Elsewhere 
among the children not in the language minority group, the difference is less than 3 percent. 
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Table 3-15. Characteristics of third-grade respondents — percent distribution by subgroup: 
School year 2001-02 







Expected 2 






Achieved 2 




Subgroup 1 


Total 


Language 

minority 


Not 

language 

minority 


Total 


Language 

minority 


Not 

language 

minority 


Total 


100.0 


100.0 


100.0 


100.0 


100.0 


100.0 


School affiliation 














Public 


78.7 


82.9 


77.1 


78.9 


83.5 


77.2 


Private 


21.3 


17.1 


22.9 


21.1 


16.5 


22.8 


Catholic 


51.6 


54.0 


50.9 


56.2 


57.4 


55.8 


Non-Catholic 


48.4 


46.0 


49.1 


43.8 


42.6 


44.2 


Type of locale 














Rural 


11.6 


5.2 


14.0 


13.1 


5.5 


15.8 


Non-rural 


88.4 


94.8 


86.0 


86.9 


94.5 


84.2 


Race/ethnicity l 3 














Hispanic 


18.7 


53.5 


5.7 


18.0 


53.4 


5.3 


Black 


14.8 


3.2 


19.2 


13.1 


2.9 


16.8 


Asian/Pacific Islander 


7.9 


23.8 


1.9 


7.7 


23.4 


2.0 


Other 


58.6 


19.6 


73.3 


61.2 


20.3 


75.9 


Race/ethnicity 2 3 














Hispanic 


18.4 


52.6 


5.5 


17.6 


52.4 


5.1 


Black 


14.8 


3.2 


19.2 


13.1 


2.9 


16.8 


Asian/Pacific Islander 


9.3 


26.1 


2.9 


9.2 


26.1 


3.1 


Other 


57.6 


18.1 


72.5 


60.1 


18.6 


75.0 



1 Characteristics of the schools (school affiliation and type of locale) are from the original sample schools. 

2 The expected sample size was computed using assumed mover rates (differential by subgroups as shown table 3-12), a 47 percent subsampling 
rate (instead of 50 percent to account for ineligibility in third grade), and a 95 percent completion rate. 

3 Race/ethnicity 1 was the strict definition of API (RACE=5 -Asian, or 6-Native Hawaiian or other Pacific Islander), while race/ethnicity 2 was the 
broader definition (RACE=5 -Asian, or 6-Native Hawaiian or other Pacific Islander, or WKASIAN=1 -Child is Asian, or WKPACISL=1 -Child is 
Native Hawaiian or other Pacific Islander)). Variables are from the ECLS-K base year data file. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2002. 
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Table 3-16 shows the third-grade data collection results by mover status (analogous to 
table 3-1 1). In this table, the total number of children is 21,357, which is larger than the total in table 3-11 
by 26 children; these are children sampled in first grade who did not have completed assessment data or 
parent interview in first grade (hence not included in table 3-11), but participated in the third-grade study 
(hence included in table 3-16). Overall the unweighted completion rate for third grade is 79 percent for 
movers and 95 percent for nonmovers, compared with the expected completion rate of 83 and 95 percent. 
The rate of base year respondents who moved out of their original sample schools is 42 percent 
(compared with the expected overall moving rate of 47 percent). The achieved sample size shown in table 
3-14 is a function of both the completion rate and the mover rate. Even though the actual completion rate 
for movers is lower than expected, the actual mover rate is also lower than expected. Fewer movers 
resulted in a larger sample size. Note that in all tables in this chapter a respondent is defined as a child 
with completed assessment data or completed parent interview data or a child who could not be assessed 
due to a disability, so that the completion rate calculated here is not the same as the completion rate in 
chapter 6 of this report or chapter 5 of the ECLS-K User’s Manual for the Third Grade Public-Use Data 
File and Electronic Code Book (NCES 2004-001) (Tourangeau, Brick, et al. 2004) which is instrument- 
specific. 
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Table 3-16. Spring-third-grade data collection results by mover status: School year 2001-02 



Spring-third grade response status Movers-sampled Nonmovers 

Unweighted Unweighted Unweighted 

Not Not completion Not Percent Percent completion Not completion 



Subgroup 1 


Total 2 


Respond 3 


respond Ineligible sampled 


rate 


Respond 3 


respond Ineligible 


moved sampled 


rate 


Respond 3 


respond 


Ineligible 


rate 


All 


21,357 


15,305 


1,524 


364 


4,164 


90.9 


3,583 


931 


361 


42.3 


53.9 


79.4 


11,722 


593 


3 


95.2 


School affiliation 


































Public 


16,784 


12,070 


1,236 


312 


3,166 


90.7 


2,726 


760 


311 


41.5 


54.5 


78.2 


9,344 


476 


1 


95.2 


Private 


4,573 


3,235 


288 


52 


998 


91.8 


857 


171 


50 


45.4 


51.9 


83.4 


2,378 


117 


2 


95.3 


Catholic 


2.356 


1,817 


115 


14 


410 


94.0 


329 


62 


14 


34.6 


49.7 


84.1 


1,488 


53 


0 


96.6 


Non-Catholic 


2,217 


1,418 


173 


38 


588 


89.1 


528 


109 


36 


56.9 


53.4 


82.9 


890 


64 


2 


93.3 


Type of locale 


































Rural 


2,478 


2,005 


162 


20 


291 


92.5 


304 


102 


19 


28.9 


59.4 


74.9 


1,701 


60 


1 


96.6 


Non-rural 


18,879 


13,300 


1,362 


344 


3,873 


90.7 


3,279 


829 


342 


44.1 


53.5 


79.8 


10,021 


533 


2 


94.9 


Race/ethnicity l 4 


































Hispanic 


3,777 


2,752 


274 


145 


606 


90.9 


756 


193 


144 


45.0 


64.3 


79.7 


1,996 


81 


1 


96.1 


Black 


3,229 


2,007 


267 


81 


874 


88.3 


604 


186 


81 


54.0 


49.9 


76.5 


1,403 


81 


0 


94.5 


Asian/Pacific Islander 
























76.4 


876 


47 


0 


94.9 




1,579 


1,174 


139 


37 


229 


89.4 


298 


92 


37 


41.5 


65.1 












Other 


12,772 


9,372 


844 


101 


2,455 


91.7 


1,925 


460 


99 


38.7 


50.3 


80.7 


7,447 


384 


2 


95.1 


Race/ethnicity 2 4 


































Hispanic 


3,698 


2,691 


271 


144 


592 


90.9 


739 


191 


143 


45.0 


64.4 


79.5 


1,952 


80 


1 


96.1 


Black 


3,229 


2,007 


267 


81 


874 


88.3 


604 


186 


81 


54.0 


49.9 


76.5 


1,403 


81 


0 


94.5 


Asian/Pacific Islander 




































1,867 


1,404 


151 


40 


272 


90.3 


351 


102 


40 


41.0 


64.4 


77.5 


1,053 


49 


0 


95.6 


Other 


12,563 


9,203 


835 


99 


2,426 


91.7 


1,889 


452 


97 


38.7 


50.1 


80.7 


7,314 


383 


2 


95.0 


Language minority 


































Non-English 


5,372 


4,041 


412 


203 


716 


90.7 


1,162 


291 


203 


44.2 


69.8 


80.0 


2,879 


121 


0 


96.0 


English 


15,985 


11,264 


1,112 


161 


3,448 


91.0 


2,421 


640 


158 


41.7 


48.3 


79.1 


8,843 


472 


3 


94.9 



1 Characteristics of the schools (school affiliation and type of locale) are from the original sample schools. 

2 The total number of children excludes 68 children who responded in fall-kindergarten and became ineligible in spring-kindergarten, and includes 165 children sampled in first grade who were eligible. 

3 A respondent is a child with assessment data or parent interview data, or a child who could not be assessed due to a disability. 

4 Race/ethnicity 1 was the strict definition of API (RACE=5 -Asian or 6-Native Hawaiian or other Pacific Islander), while race/ethnicity 2 was the broader definition (RACE=5 -Asian or 6-Native 
Hawaiian or other Pacific Islander or WKASIAN=1 -Child is Asian or WKPACISL=1 -Child is Native Hawaiian or other Pacific Islander). Variables are from the ECLS-K base year data file. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2002. 
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Spring-Fifth Grade Sample 



For the fifth-grade data collection, different options for subsampling movers were explored 
to reduce the sample size in order to contain the cost of data collection. The original plan would use the 
same procedures for third grade to subsample and follow 50 percent of children who moved in fifth grade 
or earlier and retain all language minority children who had not been subsampled out before fifth grade. 
Three alternative plans were developed to decrease the sample sizes by reducing the subsampling rates. 
One of the alternatives was adopted as the final plan. The final subsampling rates maximize the amount of 
longitudinal data available for key analytic groups. 

A new feature of the fifth-grade sample is the subsampling of children for the administration 
of the mathematics or science questionnaires. While all children retained for the fifth-grade data 
collection had child-level questionnaires filled out by their reading teachers, half were subsampled to 
have child-level questionnaires filled out by their mathematics teachers and the other half had child-level 
questionnaires filled out by their science teachers. This affects only the computation of the combined 
child-parent-teacher weights as discussed in section 7.2.4. 



3.6.1 Options for Subsampling Movers 

All sampling options considered for fifth grade were based on the beginning sample of 
21,357 children: 21,192 base year respondents who were still eligible after the base year, and 165 children 
sampled in first grade as part of the freshening procedure (see section 3.4.2). The first decision regarding 
the fifth-grade sample was to exclude the following groups of children from the fifth-grade survey, 
irrespective of other subsampling procedures that might be implemented: (1) children who had become 
ineligible in an earlier round (because they had died or moved out of the country); (2) children who were 
subsampled out in previous rounds because they had moved out of the original schools and were not 
subsampled to be followed; (3) children whose parents emphatically refused to cooperate (hard refusals) 
in any of the data collection rounds since spring-kindergarten; and (4) children eligible for the third-grade 
sample for whom there are neither first-grade nor third-grade data (i.e., no direct assessment data and no 
parent interview data from first grade and third grade). The children who met any of these conditions 
were not eligible for sampling in the fifth grade for any of the sampling options considered. In total, 5,214 
children were excluded from the fifth-grade survey; they are distributed as shown in table 3-17. 
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Table 3-17. Number of children eligible after the base year but excluded from the fifth-grade data 
collection: School year 2003-04 



Characteristics 1 


Beginning 
sample size 
after the base 
year 


Total 
number of 
children 
excluded 


Mover 
sub samp led out 
in first or third 
grade 2 


Ineligible 
in first or 
third grade 


Hard 

refusal 


Eligible for third 
grade sample, 
with no first or 
third grade data 


Total 


21,357 


5,214 


4,117 


122 


571 


404 


School affiliation 


Public 


16,771 


4,000 


3,129 


98 


433 


340 


Private 


4,570 


1,198 


988 


23 


132 


55 


Catholic 


2,354 


485 


405 


7 


52 


21 


Non-Catholic 


2,216 


713 


583 


16 


80 


34 


Unknown 


16 


16 


0 


1 


6 


9 


Type of locale 


Rural 


2,480 


381 


288 


5 


51 


37 


Non-rural 


18,733 


4,824 


3,829 


113 


518 


364 


Unknown 


144 


9 


0 


4 


2 


3 


Race/ethnicity 


Hispanic 


3,782 


811 


584 


47 


82 


98 


Black 


3,229 


1,061 


867 


12 


88 


94 


Asian/Pacific Islander 


1,580 


313 


225 


20 


46 


22 


Other 


12,678 


2,995 


2,430 


41 


343 


181 


Unknown 


88 


34 


11 


2 


12 


9 


Language minority 


Not English 


5,372 


1,000 


684 


84 


124 


108 


English 


15,985 


4,214 


3,433 


38 


447 


296 



1 Characteristics are from the most recent data available for the child (e.g., if a child was not subsampled in third grade and had data from first 
grade, then the characteristics of the child come from first grade). 

2 These are statistical movers, not operation movers as discussed in chapter 4. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



In the original plan, 50 percent of children who moved in fifth grade or earlier would be 
subsampled to be followed, and all language minority children who had not been subsampled out before 
fifth grade would be retained. This plan protects the language minority sample, as had been done in third 
grade. 



In the first alternative plan, language minority movers would be subsampled for followup 
depending on the amount of data that they had from previous rounds. If they had both spring-first grade 
and spring-third grade data, then 50 percent would be subsampled and followed if they had only one data 
point after the base year, 25 percent would be subsampled and followed. Similarly, 25 percent of other 
movers would be subsampled and followed if they had both first- and third-grade data, and 12.5 percent 
would be subsampled and followed if they had only one data point besides base year data. 
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In the second alternative plan, only children with complete longitudinal data would be 
fielded, i.e., base year respondents who had first-grade and third-grade data. Of these children, 50 percent 
of language minority children who moved in fifth grade (or earlier) would be subsampled for followup, 
and 25 percent of other movers (in fifth grade or earlier) would be subsampled for followup. Children 
who were sampled in first grade through the sample freshening procedure would not be retained in the 
sample. 



The third and last option, adopted for the fifth-grade study, called for using rates that are 
approximately equal to those given below for subsampling base year respondents who are movers in fifth 
grade (or earlier): 



■ 33 percent for non-language minority (LM) movers with full longitudinal data; 

■ 25 percent for non-LM movers with third-grade but not first-grade data; 

■ 15 percent for non-LM movers with first-grade but not third- grade data; 

■ 75 percent for LM movers with full longitudinal data; 

■ 50 percent for LM movers with third-grade but not first-grade data; and 

■ 25 percent for LM movers with first-grade but not third-grade data. 

For subsampling freshened children (i.e., children sampled in first grade) who are movers in 
fifth grade (or earlier) the rates proposed were the following: 

■ 33 percent for non-LM movers with full longitudinal data; 

■ 15 percent for non-LM movers with third-grade but not first-grade data; 

■ 15 percent for non-LM movers with first-grade but not third- grade data; 

■ 75 percent for LM movers with full longitudinal data; 

■ 25 percent for LM movers with third-grade but not first-grade data; and 

■ 25 percent for LM movers with first-grade but not third-grade data. 

Table 3-18 shows the expected fifth-grade sample size separately for language minority 
children and by mover status for the different subsampling plans, the estimated design effect due to the 
subsampling of base year children, the effective sample size, and the expected number of children with a 
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completed fifth-grade assessment, assuming that 90 percent of children would be assessed successfully. 
The completion rate of 90 percent took into account children who had moved and whether they would be 
located. According to the third-grade collection, the unweighted completion rate for child assessment was 
95 percent for nonmovers and 63 percent for movers, with an overall unweighted completion rate of 86 
percent. Since fewer movers would be included in fifth grade compared with third grade, the assumption 
was for a slightly higher completion rate for the child assessment. 



Table 3-18. ECLS-K options for subsampling movers for fifth grade: School year 2003-04 



Characteristic 


Sample size 


Design 

effect 


Effective 
sample size 


Expected number of 
children assessed 1 


Original plan 


Total 


14,135 


1.13 


12,509 


12,722 


Non-language minority: Nonmover 


4,907 


— 


4,342 


4,416 


Non-language minority: Mover 


4,856 


— 


4,297 


4,371 


Language minority: Nonmover 


1,595 


— 


1,412 


1,436 


Language minority: Mover 


2,777 


— 


2,458 


2,499 


Option 1 


Total 


11,336 


1.30 


8,720 


10,202 


Non-language minority: Nonmover 


4,908 


— 


3,775 


4,417 


Non-language minority: Mover 


3,219 


— 


2,476 


2,897 


Language minority: Nonmover 


1,595 


— 


1,227 


1,435 


Language minority: Mover 


1,614 


— 


1,242 


1,453 


Option 2 


Total 


10,308 


1.26 


8,181 


9,277 


Non-language minority: Nonmover 


4,767 


— 


3,783 


4,290 


Non-language minority: Mover 


3,067 


— 


2,434 


2,760 


Language minority: Nonmover 


768 


— 


610 


691 


Language minority: Mover 


1,706 


— 


1,354 


1,536 


Option 3 


Total 


12,635 


1.24 


10,190 


11,372 


Non-language minority: Nonmover 


4,908 


— 


3,958 


4,417 


Non-language minority: Mover 


3,784 


— 


3,052 


3,406 


Language minority: Nonmover 


1,595 


— 


1,286 


1,436 


Language minority: Mover 


2,348 


— 


1,894 


2,113 



— Not available. 

1 The expected number of children assessed is computed using an assumed completion rate of 90 percent. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 kindergarten (ECLS-K), spring 2004. 
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3.6.2 



Expected Sample Size 



Table 3-19 shows the expected sample sizes of children in fifth grade by subgroups of 
interest based on the third mover subsampling option adopted for fifth grade. The format of this table is 
different from table 3-12 for third grade. While the mover subsampling rate for third grade was constant, 
the subsampling rates vary according to the different groups described earlier. It is also difficult to 
estimate the number of schools in the fifth-grade sample because schools that were in the sample prior to 
fifth grade may drop out because of the reduction in the sample size and new schools may enter due to 
new movers. 



Table 3-19. Expected sample size for selected subgroups for fifth grade, by mover status: School year 
2003-04 



Subgroup 1 


Beginning fifth-grade sample 


After subsampling 


movers 


Total responding 


Total 


Movers 


Nonmovers 


Total 


Movers Nonmovers 


Total 


Movers Nonmovers 


All children 


16,143 


4,320 


11,823 


12,635 


6,132 


6,503 


11,372 


5,519 


5,852 


School affiliation 




















Public 


12,771 


3,340 


9,431 


10,009 


4,821 


5,187 


9,008 


4,339 


4,668 


Private 


3,372 


980 


2,392 


2,627 


1,311 


1,316 


2,364 


1,180 


1,184 


Catholic 


1,869 


373 


1,496 


1,434 


611 


823 


1,291 


550 


741 


Non-Catholic 


1,503 


607 


896 


1,193 


700 


493 


1,073 


630 


444 


Type of locale 




















Rural 


2,099 


388 


1,711 


1,558 


617 


941 


1,402 


555 


847 


Non-rural 


14,044 


3,932 


10,112 


11,077 


5,515 


5,562 


9,969 


4,964 


5,005 


Race/ethnicity 




















Hispanic 


2,971 


958 


2,013 


2,579 


1,472 


1,107 


2,321 


1,325 


996 


Black 


2,168 


757 


1,411 


1,658 


881 


776 


1,492 


793 


698 


Asian/Pacific Islander 


1,267 


382 


885 


1,104 


618 


487 


994 


556 


438 


Other 


9,737 


2,223 


7,514 


7,294 


3,161 


4,133 


6,564 


2,845 


3,719 


Language minority 




















Non-English 


4,372 


1,472 


2,900 


3,943 


2,348 


1,595 


3,549 


2,113 


1,436 


English 


11,771 


2,848 


8,923 


8,692 


3,785 


4,908 


7,823 


3,406 


4,417 



1 Characteristics of the schools (school affiliation and type of locale) are from the original sample schools. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 kindergarten (ECLS-K), spring 2004 



3.6.3 Sample Outcome and Precision Requirements 



The fifth-grade sample of 16,143 excludes the base year respondents identified in section 
3.6.1 who were not subject to data collection in the fifth grade, i.e., children who had become ineligible in 
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an earlier round (because they had died or moved out of the country); children who were subsampled out 
in previous rounds because they had moved out of the original schools and were not subsampled to be 
followed; children whose parents emphatically refused to cooperate (hard refusals) in any of the data 
collection rounds since spring-kindergarten; and children eligible for the third-grade data collection for 
whom there are neither first-grade nor third-grade data (i.e., no direct assessment data and no parent 
interview data from first grade and third grade). As discussed in section 3.1, a sample in fifth grade of 
about 10,000 children would be adequate to meet the precision requirements overall and for most 
subgroups. A sample of about 800 to 1,000 children in a subgroup would be achieved for most of the 
subgroups with an overall sample of 10,000 children and these would approximately meet the precision 
goals described in section 3.1. 

Table 3-20 (count) and table 3-21 (percent) show the characteristics of the achieved fifth- 
grade sample compared with the expected fifth-grade sample. The numbers of fifth-grade respondents for 
all of the identified subgroups of interest exceed 1,000, except for children in non-Catholic private 
schools and API children. For most of the key analytic groups the numbers of respondents are much 
larger than 1,000. For API children, the number of respondents is 970, which exceeds the minimum target 
of 800 and is very close to 1,000. The number of respondents in non-Catholic private schools is 957. The 
sample of 1 1,820 respondents attended 2,008 public schools and 356 private schools. 

Two thirds of the public schools attended by the fifth-grade sample (1,355) are transfer 
schools, and almost half of the private schools (166) are transfer schools. The large number of transfer 
schools corresponds to the heavy movement of the ECLS-K children between schools. 



3-49 




Table 3-20. Characteristics of fifth-grade respondents — number of fifth-graders by subgroup: School 
year 2003-04 



Expected 2 


Achieved 


Subgroup 1 


Total 


Language 

minority 


Not 

language 

minority 


Total 


Language 

minority 


Not 

language 

minority 


Total 


11,372 


3,549 


7,823 


11,820 


3,405 


8,415 


School affiliation 














Public 


9,008 


2,969 


6,039 


9,412 


2,845 


6,567 


Private 


2,364 


580 


1,784 


2,408 


560 


1,848 


Catholic 


1,291 


325 


966 


1,451 


330 


1,121 


Non-Catholic 


1,073 


255 


818 


957 


230 


727 


Type of locale 














Rural 


1,402 


198 


1,204 


1,659 


199 


1,460 


Non-rural 


9,969 


3,350 


6,619 


10,161 


3,206 


6,955 


Race/ethnicity 














Flispanic 


2,321 


1,894 


427 


2,244 


1,821 


423 


Black 


1,492 


108 


1,384 


1,352 


91 


1,261 


Asian/Pacific Islander 


994 


836 


158 


970 


799 


171 


Other 


6,564 


710 


5,854 


7,254 


694 


6,560 



1 Characteristics of the schools (school affiliation and type of locale) are from the original sample schools. 

2 The expected sample size was computed using assumed mover rates, differential subsampling rates, and a 90 percent completion rate. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



Table 3-21 is analogous to table 3-20 but shows the percent distribution instead of counts. 
The achieved sample is about 4 percent larger than the expected sample, with the language minority 
sample about 4 percent smaller than the expected sample. As in third grade, language minority children 
moved at a higher rate (48 percent) than non-language minority children (37 percent) resulting in the 
larger sample of non-language minority children. 

As shown in table 3-21, the agreement between the expected and achieved fifth-grade 
samples is as seen in third grade. Children in non-Catholic private schools responded at a lower rate than 
children in Catholic schools, causing the achieved sample size for children in non-Catholic schools to be 
lower than expected. For all other characteristics, the difference between expected and achieved sample 
sizes is less than 1 percent for language minority children and around 3 percent or less for non-language 
minority children. 
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Table 3-21. Characteristics of fifth-grade respondents — percent distribution by subgroup: School year 
2003-04 



Expected 2 


Achieved 


Subgroup 1 


Total 


Language 

minority 


Not 

language 

minority 


Total 


Language 

minority 


Not 

language 

minority 


Total 


100.0 


100.0 


100.0 


100.0 


100.0 


100.0 


School affiliation 














Public 


79.2 


83.7 


77.2 


79.6 


83.6 


78.0 


Private 


20.8 


16.3 


22.8 


20.4 


16.4 


22.0 


Catholic 


54.6 


56.0 


54.1 


60.3 


58.9 


60.7 


Non-Catholic 


45.4 


44.0 


45.9 


39.7 


41.1 


39.3 


Type of locale 














Rural 


12.3 


5.6 


15.4 


14.0 


5.8 


17.3 


Non-rural 


87.7 


94.4 


84.6 


86.0 


94.2 


82.7 


Race/ethnicity 














Hispanic 


20.4 


53.4 


5.5 


19.0 


53.5 


5.0 


Black 


13.1 


3.0 


17.7 


11.4 


2.7 


15.0 


Asian/Pacific Islander 


8.7 


23.6 


2.0 


8.2 


23.5 


2.0 


Other 


57.7 


20.0 


74.8 


61.4 


20.4 


78.0 



1 Characteristics of the schools (school affiliation and type of locale) are from the original sample schools. 

2 The expected sample size was computed using assumed mover rates, differential subsampling rates, and a 90 percent completion rate. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



Table 3-22 shows the details on the number of sampled students by their response and mover 
status for subgroups at the end of the fifth grade. Of the 16,143 students, 40 percent were movers and 42 
percent of the movers were subsampled for followup in fifth grade. Overall, the unweighted completion 
rate for fifth grade is 96 percent with 85 percent of movers responding and 99 percent of nonmovers 
responding. A respondent is defined as a child with completed assessment (or excluded from assessment 
due to a disability) or completed parent interview data, so that the completion rate here is not the same as 
the instrument-specific completion rate in chapter 6. 

Based on the achieved sample, the sampling and data collection procedures developed in the 
initial stages and modified throughout the course of the study did produce samples that met or exceeded 
requirements for the vast majority of key analytic groups. However, the introduction of the more intensive 
subsampling of children who moved to achieve the desired cost savings did result in some increases in the 
design effects for the estimates, as was expected. See chapter 4 of the ECLS-K Combined User’s Manual 
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for the ECLS-K Fifth-Grade Public-Use Data Files and Electronic Codebook (NCES 2006-032) 
(Tourangeau et al. forthcoming) for a discussion of design effects. 
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Table 3-22. Spring-fifth-grade data collection results by mover status: School year 2003-04 



Spring-fifth grade response rates Movers-sampled Nonmovers 



Subgroup 1 


Total 2 


Respond 2 


Not 

respond 


Not 

Ineligible sampled 


Unweighted 

completion 

rate 


Respond 3 


Not 

respond 


Ineligible 


Percent 

moved 


Percent 

sampled 


Unweighted 

completion 

rate 


Respond 3 


Not 

respond Ineligible 




All 


16,143 


11,820 


519 


41 


3,763 


95.8 


2,272 


396 


41 


40.1 


41.9 


85.2 


9,548 


123 


0 


98.7 


School affiliation 


Public 


12,771 


9,412 


433 


39 


2,887 


95.6 


1,758 


332 


39 


39.3 


42.4 


84.1 


7,654 


101 


0 


98.7 


Private 


3,372 


2,408 


86 


2 


876 


96.6 


514 


64 


2 


43.2 


39.8 


88.9 


1,894 


22 


0 


98.9 


Catholic 


1,869 


1,451 


40 


0 


378 


97.3 


234 


26 


0 


34.1 


40.8 


90.0 


1,217 


14 


0 


98.9 


Non-Catholic 


1,503 


957 


46 


2 


498 


95.4 


280 


38 


2 


54.4 


39.1 


88.1 


677 


8 


0 


98.8 


Type of locale 


Rural 


2,099 


1,659 


44 


2 


394 


97.4 


157 


36 


2 


28.1 


33.1 


81.3 


1,502 


8 


0 


99.5 


Non-rural 


14,044 


10,161 


475 


39 


3,369 


95.5 


2,115 


360 


39 


41.9 


42.7 


85.5 


8,046 


115 


0 


98.6 


Race/ethnicity 1 


Hispanic 


2,971 


2,244 


136 


14 


577 


94.3 


658 


127 


14 


46.3 


58.1 


83.8 


1,586 


9 


0 


99.4 


Black 


2,168 


1,352 


69 


4 


743 


95.1 


325 


58 


4 


52.1 


34.2 


84.9 


1,027 


11 


0 


98.9 


Asian/Pacific Islander 


1,267 


970 


74 


10 


213 


92.9 


293 


62 


10 


45.6 


63.1 


82.5 


677 


12 


0 


98.3 


Other 


9,737 


7,254 


240 


13 


2,230 


96.8 


996 


149 


13 


34.8 


34.2 


87.0 


6,258 


91 


0 


98.6 


Language minority 


Non-English 


4,372 


3,405 


242 


33 


692 


93.4 


1,140 


222 


33 


47.7 


66.8 


83.7 


2,265 


20 


0 


99.1 


English 


11,771 


8,415 


277 


8 


3,071 


96.8 


1,132 


174 


8 


37.3 


30.0 


86.7 


7,283 


103 


0 


98.6 



1 Characteristics of the schools (school affiliation and type of locale) are from the original sample schools. 

2 A respondent is a child with assessment data or parent interview data, or a child who could not be assessed due to a disability. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 
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4. DATA COLLECTION METHODS 



The following sections discuss the data collection procedures in the fifth-grade data 
collection phase of the Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K). 
Section 4.1 gives an overview of the data collection methods. Detailed information is provided on roles 
and responsibilities in the study (section 4.2), study training procedures (section 4.3), fall preassessment 
school contacts (section 4.4), spring-fifth grade data collection (section 4.5), and data collection quality 
control procedures (section 4.6). 



4.1 Overview of Data Collection Methods 

The ECLS-K fifth-grade data collection was conducted in the fall and spring of the 2003-04 
school year. Fall data collection included contacting sampled schools to schedule appointments to conduct 
the child assessments in the spring of the school year, verify the parent consent procedures, link children 
to their teachers, identify children who had withdrawn from the school, and obtain locating information 
about their new schools. Spring data collection instruments included the administration of direct child 
assessments and parent interviews and the collection of teacher and school questionnaires, student record 
abstracts, and facilities checklists. The activities to locate children and gain cooperation of the schools 
into which they transferred began in the fall and continued during the spring data collection. The content 
and timeline of the fifth-grade data collection is shown in exhibit 4-1. 

The mode of data collection was computer-assisted personal interviewing (CAPI) for the 
child assessments; telephone and in-person computer-assisted interviewing (CAI) was the mode of data 
collection for the parent interview; and self-administered questionnaires were used to gather information 
from teachers, school administrators, and student records. Field staff completed the facilities checklist. 



4-1 




Exhibit 4-1. Timeline of fifth-grade data collection: School year 2003-04 





Fifth grade 




2003 




2004 


Fall 


Winter 


Spring 


Advance school 


Advance school 


Child assessments 


contact 

| 


contact 

| 


conducted 

| 


1 

Tracing sampled 


1 

Tracing sampled 


1 

Parent interviews 


households 


households 


conducted 

1 

Teacher 

information 

collected 

1 

School and school 
administrator data, 
facilities checklist 
collected 

1 

Student records data 
collected 

1 

Tracing children 
who moved 

1 

Activities to gain transfer 
school cooperation 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



4.2 Roles and Responsibilities in the ECLS-K Study 

4.2.1 School’s Role 



During school recruitment, the schools were asked to designate a staff member to be the 
school coordinator to assist the ECLS-K staff with all school arrangements. 

Since the child assessments were administered at the schools, schools needed to provide 
appropriate space for conducting the assessments. 
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4.2.2 



School Coordinator’s Role 



A school coordinator was designated by the principal to facilitate the ECLS-K activities in 
the school. The school coordinator played a significant role in the smooth functioning and successful 
completion of the ECLS-K child assessments in each cooperating school. Lie or she knew the personality 
of the school, the most opportune times to schedule the assessments, the available locations where the 
one-on-one assessments could be conducted, and the best way to notify students, their parents, and their 
teachers of the assessment. 

The coordinator was asked to assist the ECLS-K in four ways: 

■ Notify selected students, their teachers, and their parents of the study; 

■ Arrange for suitable space for the assessment activities; 

■ Provide information on sampled children, such as their grade and teachers’ names; 
and 

■ Distribute teacher and school questionnaires. 



4.2.3 Supervisor’s Role 

There were a total of 8 1 supervisors during the fifth-grade data collection who oversaw field 
staff conducting both the parent interviews and child assessments. The supervisors’ responsibilities were 
as follows: 



■ Contact each school assigned to them to 

1. Arrange for space to conduct the assessment; 

2. Schedule the spring assessment in original schools; 

3. Discuss parental contact information and consent procedures; 

4. Link children to teachers and domains; and 

5. Identify children who had withdrawn from the school; 

■ Follow up and track receipt of parental consent forms, as necessary; 

■ Update the Field Management System (FMS) regularly and report to their field 
manager; 
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■ Transmit updated FMS data to the home office; 

■ Pick up e-mail regularly; and 

■ Return all materials at the end of the field period. 



4.2.4 Assessor’s Role 

A team of three to four assessors worked with each supervisor in a work area. The primary 
responsibilities of the ECLS-K assessors were to conduct the computer-assisted one-on-one child 
assessments and parent interviews. There were a total of 258 assessors, with 16 having conducted the 
parent telephone interview only (these 1 6 assessors are referred to as interviewers rather than assessors in 
the remainder of this chapter). The remaining 242 conducted both the parent interview and the child 
assessments. 



In addition to these responsibilities, some assessors were asked by their supervisor to assist 
with various other activities that took place in the school. These included, but were not limited to, 
preparing parental consent forms (if required), collecting teacher questionnaires, and assisting with 
various other recordkeeping tasks. 



4.2.5 Field Manager’s Role 

Six experienced regional field managers were assigned to oversee the work of the 81 
supervisors. The field managers held weekly telephone conference calls with each supervisor assigned to 
them. If a supervisor had an immediate problem, he or she was encouraged to call the field manager at 
any time. 



Depending on the stage of the field period, the telephone conference calls between 
supervisors and field managers reviewed those activities that were in the planning stage, in progress, or in 
the process of being completed. These discussions included the following topics: 

■ Status of telephone contacts with original and transfer schools; 

■ Status of assessments scheduled in original schools by work area; 
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■ Status of parent consent and followup in original and transfer schools; 

■ Status of linking children to teachers; 

■ Status of children who were withdrawn from the school; 

■ Any refusal cases; 

■ Receipt of all school materials; and 

■ Overall and individual costs in the work area. 



Exhibit 4-2 presents the fifth-grade field organization and the number of staff in each 

position. 

Exhibit 4-2. Fifth-grade field organization 




SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



4.3 Field Staff Training 

Several in-person training sessions were conducted to prepare staff for the fifth-grade data 
collection. In the fall of 2003, supervisors were trained to contact original schools and recruit transfer 
schools. In the spring of 2004, three training sessions were held: one to train trainers, one for field 
supervisors, and one for assessors. All training sessions were conducted using scripted training manuals 
to ensure that all trainees received the same information. Training sessions consisted of interactive 
lectures, scripted role plays, interactive exercises, and self-administered exercises. Interactive lectures 
were lectures with discussion and questions occurring periodically during the lecture. Scripted role plays 
usually consisted of pairs of trainees each pretending to be, for example, the assessor and the child or the 
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interviewer and the parent. Such role plays gave trainees a chance to become more familiar with their 
materials and duties. Interactive exercises were group exercises led by the trainer in which all trainees 
participated. Self-administered exercises were, as the name suggests, completed by trainees working 
independently. Because of the complexity of the ECLS-K, trainees were required to become familiar with 
the functionality of their laptop computers and with the programs installed on them. Trainees were also 
required to become familiar with the different child assessment materials. See chapter 2 of the ECLS-K 
Psychometric Report for the Third Grade (NCES 2005-062) (Pollack et al. 2005) for a description of the 
child assessment materials. The following sections discuss the fall and spring trainings. 



4.3.1 Advance Contact and Recruitment Training 

During the fall 2003, advance contact was made with the schools in order to remind them 
about the study and to collect information that would be helpful in the spring 2004 national data 
collection. The advance effort not only reduced the burden on supervisors in the spring but also 
reacquainted schools with the study’s procedures and gave supervisors a chance to encourage the schools’ 
participation. The major fall tasks were to contact schools to set appointments for the child assessments in 
the spring, to verify the parent consent procedures, to link children to teachers and domains, and to 
identify children who had withdrawn from the school and obtain locating information about their new 
schools. Field supervisors were trained for 3 days in September 2003 to contact original sampled schools 
and transfer schools to set up the data collection in the spring. A total of 39 field supervisors and 2 field 
managers completed advance contact and recruitment training. Topics included an overview of study 
activities to date, verifying parent consent procedures, identifying and locating children who moved from 
the schools they attended in the first grade, identifying the teachers of ECLS-K children and linking them 
to those children, and exercises on scheduling schools efficiently within a work area. 

As in the third-grade training, advance contact and recruitment training was conducted using 
the automated FMS. The FMS is a database that is used throughout the data collection period to enter 
information about the sampled children, parents, teachers, and school and to monitor production on all 
data collection activities. The FMS contains information essential to conducting and monitoring the 
progress of the data collection. For example, it contains the names and addresses of each school, the 
principal’s name and telephone number, the name and telephone number of the school coordinator, the 
first and last day of classes, the school hours, and the names of the sampled children in the school. For 
children, it contains information on their names and their parents’ names, whether they have any 
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accommodations or need to use a hearing aid or glasses, as well as other information. Information of the 
same sort is also collected about the teachers and parents. For example, the FMS contains the name, 
address, and telephone number of each parent. For each parent, the name of the sampled child (or 
children) is also listed. For teachers, the FMS contains their names, schools, and the sampled children in 
their classrooms. It also indicates whether the teacher was a regular or a special education teacher, and 
what subject the regular teacher taught and in how many classrooms. To monitor production, the FMS 
contains case disposition codes and case assignment information. During training presentations, the field 
supervisors entered information into the FMS, thus acquiring hands-on experience with the FMS and all 
field procedures prior to beginning data collection. The field supervisors also completed role-play 
exercises that involved entering information into the FMS. 



4.3.2 Spring-Fifth Grade Training 

Field supervisors, interviewers, and assessors were trained for the spring-fifth grade data 
collection in two sessions in February 2004. The first session was trainers’ training and certification. The 
second and largest training involved the training of the supervisors and assessors. Before the February in- 
person training session, supervisors and assessors completed 8 hours of home study training on the study 
design, field procedures, and computer keyboard skills. Staff conducting only the parent interviews did 
not have a pre-session home study to complete. 



4.3.2. 1 Trainers’ Training and Certification 

The purpose of trainers’ training was to (1) introduce lead and co-trainers to the training 
materials; (2) evaluate the flow, language, exercises, and time allotment of the training sessions; and 
(3) certify trainers on the child assessment. Some, but not all, co-trainers and runners (staff who assisted 
trainees who experienced difficulties with the CAI application and helped with the management and 
distribution of training materials) were also certified on the child assessment. Experienced trainers with 
in-depth knowledge of the ECLS-K conducted the trainer training sessions. Not only had these trainers 
developed the CAI specs, but they had also worked with expert consultants to develop the child 
assessments and the assessment materials. In addition, they conducted nine assessments on nonsampled 
children in October 2003 in order to become certified on the assessments of the ECLS-K prior to the 
launch of the national data collection (see section 4. 3. 2.2). 
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Approximately 16 lead trainers, 16 co-trainers, and 16 runners were trained at trainers’ 
training in a single room. As noted earlier, experienced trainers conducted the parent interview and the 
child assessment training sessions. In addition, a data display person responsible for running the 
electronic data display and two runners assisted in the training. Trainers’ training was conducted for 4 
days in January 2004 in Rockville, Maryland. The trainers’ training agenda covered many of the same 
topics that were presented during the assessment and parent interview training for the national data 
collection, as well as some additional topics on the trainers’ tasks and responsibilities. Exhibit 4-3 is the 
trainers’ training agenda. In addition, trainers were certified on the child assessment following the 
procedures described in section 4.3.2. 2. The only difference between trainer certification and field staff 
certification was that trainers were certified on an entire child assessment and the field staff were certified 
on approximately half of a child assessment. 



Exhibit 4-3. Spring-fifth grade Trainer Training Agenda: School year 2003-04 



Day 


Time 


Session # 


Topic 


Type of session 


Trainer 


On Computer? 


1 


1/28/2004 














9:00-9:30 


1 


Study Introduction 


Interactive Lecture 


Lead Trainer 


No 




9:30-10:00 


2 


Parent Interview 
Introduction 


Interactive Lecture 


Lead Trainer 


Yes 








BREAK 










10:15-12:15 


3 


Parent Interactive #1 


Interactive Role Play 


Lead Trainer 


Yes 




12:15-1:30 




LUNCH 










1:30-2:30 


5 


Parent Interact #2 
(with Twins) 


Interactive Role Play 


Co-Trainer 


Yes 




2:30-2:45 




BREAK 










2:45-3:45 


5 


Parent Interact #2 
(with Twins) (con’t) 


Interactive Role Play 


Co-Trainer 


Yes 




3:45-5:00 


8 


Child Assessment and 

Certification 

Overview 


Interactive Lecture 


Lead Trainer 


No 


2 


1/29/2004 














9:00-10:00 


9 


Introduction and Self- 
description 
Questionnaire 


Interactive Lecture 


Lead Trainer 


Yes 




10:00-10:15 


10 


Reading — Routing 
and Low Form 


Interactive Lecture 


Lead Trainer 


Yes 



See note at end of exhibit. 
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Exhibit 4-3. Spring-fifth grade Trainer Training Agenda: School year 2003-04 — Continued 



Day 


Time 


Session # 


Topic 


Type of session 


Trainer 


On Computer? 




10:15-10:30 




BREAK 










10:30-11:00 


10 


Reading — Routing 
and Low Form (con’t) 


Interactive Lecture 


Lead Trainer 


Yes 




11:00-11:15 


11 


Reading — Routing 
and Low Form 
(practice/exercises) 


Individual Practice 




Yes 




11:15-12:00 


12 


Reading — Routing 
and Middle Form 


Interactive Lecture 


Co-Trainer 


Yes 




12:00-1:00 




LUNCH 










1:00-1:15 


13 


Reading — Routing 
and Middle Form 
(practice/exercises) 


Individual Practice 




Yes 




1:45-2:00 


15 


Reading — High Form 
(practice/exercises) 


Individual Practice 




Yes 




2:00-2:45 


16 


Math — Routing and 
Low Form 


Interactive Lecture 


Lead Trainer 


Yes 




2:45-3:00 


17 


Math — Routing and 
Low Form 
(practice/exercises) 


Individual Practice 




Yes 




3:00-3:15 




BREAK 










3:15-4:00 


18 


Math — Routing and 
Middle Form 


Interactive Lecture 


Co-Trainer 


Yes 




4:00-4:15 


19 


Math — Routing and 
Middle Form 
(practice/exercises) 


Individual Practice 




Yes 




4:15-4:45 


20 


Math — High Form 


Interactive Lecture 


Lead Trainer 


Yes 




4:45-5:00 


21 


Math — High Form 
(practice/exercises) 


Individual Practice 




Yes 


3 


1/30/2004 














9:00-9:45 


22 


Science — Routing and 
Low Form 


Interactive Lecture 


Lead Trainer 


Yes 




9:45-10:00 


23 


Science — Routing and 
Low Form 
(practice/exercises) 


Individual Practice 


Lead Trainer 


Yes 




10:00-10:15 




BREAK 










10:15-11:00 


24 


Science — Routing and 
Middle Form 


Interactive Lecture 


Co-Trainer 


Yes 




11:00-11:15 


25 


Science — Routing and 
Middle Form 
(practice/exercises) 


Individual Practice 


Lead Trainer 


Yes 




11:15-11:45 


26 


Science — High Form 


Interactive Lecture 


Lead Trainer 


Yes 



See note at end of exhibit. 
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Exhibit 4-3. Spring-fifth grade Trainer Training Agenda: School year 2003-04 — Continued 



Day 


Time 


Session # 


Topic 


Type of session 


Trainer 


On Computer? 




11:45-12:00 


27 


Science — High Form 
(Practice/exercises) 


Individual Practice 




Yes 




12:00-1:00 




LUNCH 










1:00-1:30 


28 


Food Consumption 
Questionnaire 


Interactive Lecture 


Co-Trainer 


Yes 




1:30-2:00 


29 


Height and Weight 
Lecture 




Co-Trainer 






2:00-2:30 


30 


Height and Weight 
Practice 


Individual Practice 




Yes 




2:30-3:00 


31 


Closing Interactive 


Interactive Lecture 


Lead Trainer 


Yes 




3:00-3:15 




BREAK 










3:15-4:45 


32 


Full Child Interactive 


Interactive Lecture 


Lead Trainer 


Yes 




4:45-5:15 


33 


Maximizing the 
Child’s Performance 


Interactive Lecture 


Co-Trainer 


No 



Day 


Time 


Session # 


Topic 


Type of session 


Trainer 


On Computer? 


4 


1/31/2004 














9:00-10:00 




Training 

responsibilities 


Discussion 




No 




10:00-12:00 




Wave 1: Child 
respondents 






No 




12:00-1:00 




LUNCH 






No 




1:00-3:00 




Wave 2: Child 
respondents 






No 




3:00-5:00 




Wave 3: Child 
respondents 






No 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



Field Supervisor Training. Field supervisors had 3 days of training before the assessor 
training in Los Angeles, California. New supervisors for spring data collection attended the first and 
second day of training and all supervisors, including those who conducted fall advance school contacts, 
attended the third day. Field supervisors were also trained to use the FMS, and the field supervisors 
entered information into the FMS during training presentations. Eighty-one field supervisors completed 
training. The topics covered in the field supervisor training session included reviewing materials from the 
fall school contact and recruitment, role playing to practice contacting school coordinators; identifying 
and locating children who moved from their third-grade schools, identifying the regular and special 
education teachers of ECLS-K children and linking them to those children and domains, distributing and 
following up on teacher and school administrator questionnaires, completing the facilities checklist, and 
conducting quality control observations. Exhibit 4-4 presents the field supervisor training agenda. 
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Exhibit 4-4. Spring-fifth grade field supervisor training agenda: School year 2003-04 



ECLS-K Round 5 Training 



National Supervisor Training 



Day Time 



2/13/05 



8:30-9:00 



9:00-9:45 



9:45-10:15 



10:15-10:30 



10:30-12:00 



12 : 00 - 1:00 



1:00-1:45 



1:45-2:30 



2:30-3:15 



3:15-3:30 



3:30-4:30 



4:30-5:30 



2/14/04 



8:30-9:30 



9:30-10:30 



10:30-10:45 



10:45-11:15 



11:15-12:00 



12 : 00 - 1:00 



1:00-1:45 



1:45-3:30 



3:30-3:45 



3:45-4:30 



4:30-5:30 



Session # 



Topic 



Type of session 












Study Overview 



School Contacts and Materials 



Advance School Contact 
Activities 



BREAK 



Call to the School Coordinator 



LUNCH 



School Coordinator Exercises 



School Coordinator Role Plays 



Additional Preassessment 
Activities 



BREAK 



Identifying Transfer Children 



Identifying Transfer Children’ 
Exercises 



Assessment Day 
Responsibilities 



School/Teacher Questionnaires 



BREAK 



School/Teacher Questionnaires 
Exercises 



Strategies for Obtaining 
Completed School/Teacher 
Questionnaires 



LUNCH 



Strategies for Obtaining 
Completed School/Teacher 
Questionnaires Role Plays 



Refielding Transfer Children 



BREAK 



Refielding Transfer Children’ 
Exercises 



Solo Practice on the FMS 



Interactive Lecture 



Interactive Lecture 



Interactive Lecture 




Interactive Exercises 



Dyad Role Plays 



Interactive Lecture 



Interactive Lecture 



Interactive Exercises 



Interactive Lecture 



Interactive Lecture 



Interactive Exercises 



Interactive Lecture 



Dyad Role Plays 



Interactive Lecture 



Interactive Exercises 



Self-administered Exercises 
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Exhibit 4-4. Spring-fifth grade field supervisor training agenda: School year 2003-04 — Continued 



National Supervisor Training 


Day 


Time 


Session # 


Topic 


Type of session 


3 


2/15/04 










8:30-9:30 


18 


Case Management 


Interactive Lecture 




9:30-10:30 


19 


Parent Interview Field Issues 


Interactive Lecture 




10:30-10:45 




BREAK 






10:45-11:15 


20 


Refusal Conversion Workshop 


Interactive Lecture 




11:15-12:00 


21 


Refusal Conversion Role Plays 


Dyad Role Plays 




12:00-1:00 




LUNCH 






1:00-1:45 


22 


Quality Control 


Interactive Lecture 




1:45-2:30 


23 


Quality Control Exercises 


Interactive Exercises 




2:30-3:00 


24 


Assigning Disposition Codes 


Interactive Lecture 




3:00-3:15 




BREAK 






3:15-4:00 


25 


Assigning Disposition Codes 
Exercises 


Interactive Exercises 




4:00-5:30 


26 


Administrative Issues 


Interactive Lecture 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



Assessor Training. The assessor training sessions were conducted in Los Angeles, 
California. Assessor training lasted for 5 days; field supervisors were also trained to perform all assessor 
activities; bilingual assessors were trained for half a day to conduct the parent interview in Spanish. Two 
hundred forty-two assessors, 16 parent interviewers, and 81 field supervisors completed training. 
Assessor training included an overview of study activities to date, interactive lectures on the direct child 
assessments and the parent interview, role-play scripts to practice parent interviews and direct child 
assessments, direct child assessment precertification exercises on each form of the direct child domain 
assessments, techniques for parent refusal avoidance, and strategies for building rapport with children. A 
major goal of the assessor training was to train field staff in the proper procedures to conduct the direct 
child assessments. The sessions provided trainees with practical experience with all the direct child 
assessment materials and procedures and the CAI programs before data collection. Trainees practiced 
entering information into the CAI system on laptop computers during training sessions on conducting the 
direct child assessments and parent interview role-play scripts. Exhibit 4-5 presents the assessor training 
agenda. 
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Exhibit 4-5. Spring-fifth grade assessor training: School year 2003-04 



Day 


Time 


Session # 


Topic 


Type of session 


Trainer 


On Computer? 


1 


2/17/2004 














8:30-9:00 


1 


Study Introduction 


Interactive Lecture 


Lead Trainer 


No 




9:00-9:30 


2 


Parent Interview Introduction 


Interactive Lecture 


Lead Trainer 


Yes 




9:30-11:45 


3 


Parent Interactive #1 


Interactive Role Play 


Lead Trainer 


Yes 




10:00-10:15 




BREAK 










11:45-1:00 




LUNCH (includes Welcome) 










1:00-2:00 


4 


Parent Dyad #1 


Dyad Role Play 


Lead Trainer 


Yes 




2:00-3:00 


5 


Parent Interact #2 (with Twins) 


Interactive Role Play 


Co-Trainer 


Yes 




3:00-3:15 




BREAK 










3:15-5:30 


6 


Parent Dyad #2 


Dyad Role Plays 


Lead Trainer 


Yes 



See note at end of exhibit. 
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Exhibit 4-5. Spring-fifth grade assessor training: School year 2003-04 — Continued 



Day 


Time 


Session # 


Topic 


Type of session 


Trainer 


On Computer? 


2 


2/18/2004 














8:30-9:45 


7 


Final evaluation role play 


Dyad Role Plays 


Lead Trainer 


Yes 




9:45-10:00 




BREAK 




Lead Trainer 






10:00-11:15 


8 


Child Assessment and 
Certification Overview 


Interactive Lecture 


Lead Trainer 


No 




11:15-12:15 


9 


Introduction and Self- 
description Questionnaire 


Interactive Lecture 


Lead Trainer 


Yes 




12:15-1:15 




LUNCH 










1:15-2:00 


10 


Reading — Routing and Low 
Form 


Interactive Lecture 


Lead Trainer 


Yes 




2:00-2:45 


11 


Reading — Routing and Low 
Form (practice/exercises) 


Individual Practice 




Yes 




2:45-3:30 


12 


Reading — Routing and 
Middle Form 


Interactive Lecture 


Co-Trainer 


Yes 




3:00-3:15 




BREAK 










3:30-4:15 


13 


Reading — Routing and 
Middle Form 
(practice/exercises) 


Individual Practice 




Yes 




4:15-4:45 


14 


Reading — High Form 


Interactive Lecture 


Lead Trainer 


Yes 




4:45-5:30 


15 


Reading — High Form 
(practice/exercises) 


Individual Practice 




Yes 



See note at end of exhibit. 
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Exhibit 4-5. Spring-fifth grade assessor training: School year 2003-04 — Continued 



Day 


Time 


Session # 


Topic 


Type of session 


Trainer 


On Computer? 


3 


2/19/2004 














8:30-9:15 


16 


Math — Routing and Low 
Form 


Interactive Lecture 


Lead Trainer 


Yes 




9:15-10:00 


17 


Math — Routing and Low 
Form (practice/exercises) 


Individual Practice 




Yes 




10:00-10:15 




BREAK 










10:15-11:00 


18 


Math — Routing and Middle 
Form 


Interactive Lecture 


Co-Trainer 


Yes 




11:00-11:45 


19 


Math — Routing and Middle 
Form (practice/exercises) 


Individual Practice 




Yes 




11:45-12:15 


20 


Math — Fligh Form 


Interactive Lecture 


Lead Trainer 


Yes 




12:15-1:15 




LUNCH 










1:15-2:30 


21 


Math — High Form 
(practice/exercises) 


Individual Practice 




Yes 




2:30-3:15 


22 


Science — Routing and Low 
Form 


Interactive Lecture 


Lead Trainer 


Yes 




3:15-3:30 




BREAK 










3:30-4:15 


23 


Science — Routing and Low 
Form (practice/exercises) 


Individual Practice 


Lead Trainer 


Yes 




4:15-4:45 


24 


Science — Routing and Middle 
Form 


Interactive Lecture 


Co-Trainer 


Yes 




4:45-5:30 


25 


Science — Routing and Middle 
Form (practice/exercises) 


Individual Practice 


Lead Trainer 


Yes 



See note at end of exhibit. 
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Exhibit 4-5. Spring-fifth grade assessor training: School year 2003-04 — Continued 



Day 


Time 


Session # 


Topic 


Type of session 


Trainer 


On Computer? 


4 


2/20/2004 














8:30-9:00 


26 


Science — High Form 


Interactive Lecture 


Lead Trainer 


Yes 




9:00-9:45 


27 


Science — High Form 
(Practice/exercises) 


Individual Practice 




Yes 




9:45-10:15 


28 


Food Consumption 
Questionnaire 


Interactive Lecture 


Co-Trainer 


Yes 




10:15-10:30 




BREAK 










10:30-11:00 


29 


Height and Weight Lecture 




Co-Trainer 






11:00-11:30 




Height and Weight Practice 


Individual Practice 




Yes 




11:30-12:15 


31 


Closing Interactive 


Interactive Lecture 


Lead Trainer 


Yes 




12:15-1:15 




LUNCH 










1:15-3:00 


32 


Full Child Interactive 


Interactive Lecture 


Lead Trainer 


Yes 




3:00-3:15 




BREAK 










3:15-3:45 


33 


Maximizing the Child’s 
Performance 


Interactive Lecture 


Co-Trainer 


No 




3:45-5:30 


34/35 


Full Child Role Plays 1 & 2 


Dyad Role Plays 




Yes 



See note at end of exhibit. 
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Exhibit 4-5. Spring-fifth grade assessor training: School year 2003-04 — Continued 



Day 


Time 


Session # 


Topic 


Type of session 


Trainer 


On Computer? 


5 


2/21/2004 














8:30-8:45 


36 


Transmission 


Interactive Lecture 


Lead Trainer 


Yes 




8:45-9:45 


37 


Full Child Practice 


Individual Practice 




Yes 




9:45-10:00 




BREAK 










10:00-11:15 


W1 


Wave 1 Live Respondents 


Observation 




Yes 








CAPILAB/Review Assignments 


Workshop 








11:30-12:45 


W2 


Wave 2 Live Respondents 


Observation 




Yes 








CAPILAB/Review Assignments 


Workshop 












Live Assessment Debriefing 


Discussion 








11:30-2:30 




LUNCH 










1:00-2:15 


W3 


Wave 3 Live Respondents 


Observation 




Yes 








CAPILAB/Review Assignments 


Workshop 












Live Assessment Debriefing 


Discussion 








2:30-3:45 


W4 


Wave 4 Live Respondents 


Observation 




Yes 








CAPILAB/Review Assignments 


Workshop 












Live Assessment Debriefing 


Discussion 







See note at end of exhibit. 
















































Exhibit 4-5. Spring-fifth grade assessor training: School year 2003-04 — Continued 



Day 


Time 


Session # 


Topic 


Type of session 


Trainer 


On Computer? 


6 


2/22/2004 




SPANISH TRAINING 










8:30-9:30 


38 


Gaining Cooperation 


Interactive Role Play 


Lead Trainer 


No 




9:30-11:00 


39 


Parent Interactive #1 


Interactive Role Play 


Lead Trainer 


No 




10:15-10:30 




BREAK 










11:00-12:00 


40 


Parent Dyad #1 


Dyad Role Play 




No 




12:00-1:00 




LUNCH 










1:00-2:00 


40 


Parent Dyad #2 


Dyad Role Play 


Lead Trainer 


No 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 











































43.2.2 



Certification of the Child Assessors 



Assessors and field supervisors conducted child assessments. Training to administer the 
assessment battery included exercises that took into account the different ways children may answer 
questions. Ambiguous answers were included in both training interactives and role plays to provide 
practice in coding a variety of responses. Particular attention was paid to the question-by-question 
specifications (QxQs) for constructed response/open-ended questions so that a clear understanding of 
coding guidelines was established. These guidelines were reviewed frequently during training so that 
trainees were prepared for answers that were not phrased exactly as found in the scoring rubrics. 

Training on the specific subdomains was implemented in four parts. The first two parts 
involved precertification activities and the second two parts involved certification activities. The 
precertification activities involved an interactive lecture in which the trainer reviewed the specific items 
within the subdomain (i.e., reading, mathematics, and science test forms) using the QxQs and a 
standardized training script and individual practice, which addressed both item administration skills and 
accurate coding of responses. The certification activities involved written certification exercises and a live 
certification session during which each trainee conducted a portion of the assessment with an actual child 
respondent. 



The first part of the precertification training for each subdomain was an interactive lecture 
using the QxQs, which highlighted the constructed response/open-ended assessment items. Although 
conceivably children could produce a variety of answers for any item, there are only 54 questions (25 
percent of the items) in the entire assessment that were open-ended items and would be expected to have 
the most variation. A constructed response/open-ended question is a short answer question that is scored 
against a specific criterion or a scoring rubric (scoring guide). These types of questions require the 
assessor to inteipret and use multiple criteria in evaluating the responses. The assessor compared a child’s 
response to a scoring rubric in order to determine if the child’s response was correct or incorrect. Most of 
the open-ended assessment items were in the reading and science domains. 

The second part of the precertification training involved independent review and individual 
practice in administering the assessment domain. Once an interactive lecture was completed, the trainees 
practiced administering the subdomain that had just been presented. The practice was not scripted. 
However, trainees were instructed to concentrate on practicing item administration skills, such as reading 
verbatim from the laptop computer screen and gesturing. Trainers observed trainees during the individual 
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practice, recorded the results of the individual practice in the Trainee Evaluation Form and gave each 
trainee specific feedback on performance. The trainer reviewed the item administration skills with the 
trainee and provided additional instruction to trainees requiring additional practice. 

The third part of the training on each subdomain involved written certification exercises. 
These exercises concentrated on the open-ended questions and were designed to evaluate the trainees’ 
ability to accurately select the appropriate response codes. Each exercise presented a constructed 
response/open-ended question with a variety of responses (some from the scoring rubric as well as some 
variations on those), and the assessor was required to code each response as either correct or incorrect. 
Training staff scored these exercises and feedback was given to the trainee on coding accuracy. Most 
trainees passed the written exercises on their first attempt. Less than a quarter of the trainees (77 trainees 
or 24 percent) did not pass at least one element of the reading certification exercises on the first attempt. 
The mathematics and science certification exercises were considerably easier for trainees; only two 
trainees were required to repeat any mathematics exercise and 55 trainees (17 percent) were required to 
repeat the science yellow certification exercise. This variability was due to the complexity of the fifth- 
grade reading scoring rubrics and the unfamiliarity of the exercises themselves (reading exercises were 
distributed first, with mathematics and science exercises on later days). The trainees who did not pass the 
written certification exercises completed specified remedial training steps prescribed by training staff. All 
of the trainees who had to re-take the exercises after the remedial evening session achieved a passing 
score. No field staff were released because of failure on the certification exercises. 

Training staff implemented a variety of remedial training steps for individual trainees and 
trainees collectively. The QxQs and interactive lectures were reviewed as necessary and the trainee(s) 
practiced the subdomain test again. The help laboratory was made available after the daily training 
session to provide additional instruction and help to trainees before the live portion of the child 
assessment certification. 

The culmination of the child assessments training was administering the cognitive 
assessment battery to children. This allowed the trainees to experience what it is like in the field and to 
put into practice all their new skills. Training staff who were already certified on the assessment observed 
trainees as they administered parts (e.g., a routing test and a level test) of the assessment to fifth grade- 
aged children. Feedback on trainee performance was given using the Assessment Certification Form. 
Supervisors and assessors were certified on the child assessment by administering approximately half of a 
cognitive assessment to a child while being observed by certified evaluators. As evaluators observed the 
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assessment, they completed the appropriate sections of the Assessment Certification Form. This form had 
both the trainee and the evaluator’s names recorded on the cover as well as the date the evaluation was 
conducted. 



The Assessment Certification Form had two sections: section 1 (exhibit 4-6) rated the trainee 
on key skill areas, such as building rapport, using neutral praise, responding to behaviors presented by the 
child, appropriate pacing, and avoiding coaching. The evaluator marked each skill area that the trainee did 
not demonstrate appropriately. 



Exhibit 4-6. Section 1 of the Assessment Certification Form: School year 2003-04 



Evaluator: As the assessment is administered, record whether or not the assessor successfully 
performed the following behaviors. Check “No” if the assessor makes 3 or 4 errors and needs to 
make improvements. 



SECTION 1 : Rapport building and working with the child 



Q: 


Did the assessor... 


No 


1 . 


Establish rapport with the child? 




2. 


Use appropriate response to DK responses from the child? 




3. 


Use neutral praise? 




4. 


Respond to behaviors presented by child? 




5. 


Avoid coaching the child? 




6. 







SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



Section 2 listed specific questions from each routing and subdomain form. The instructions 
for completing section 2 are shown in exhibit 4-7. For each of the listed questions observed, the evaluator 
recorded both the child’s response and noted if the trainee did not demonstrate the specified required 
administration skills for that question. The required administration skills included reading questions 
verbatim, using appropriate probes, and using appropriate hand motions (gesturing). For each question on 
which the evaluator observed that the trainee did not demonstrate the required administration skill(s), he 
or she checked a box, indicating which skill was not performed. 
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Exhibit 4-7. Instructions for Section 2 of the Assessment Certification Form: School year 2003-04 

SECTION 2 : Specific Assessment Activities 

Supervisor/Evaluator: Code the items as the assessor administers the assessment. 

Code the child’s response as the item is administered. 

If the item requires probing, check the box if the assessor does not use the appropriate probe. 

Check the box in the “Verbatim” column if the assessor does not read the item exactly as worded on the 
screen. 

Check the box in the “Gesturing” column if the assessor does not use appropriate hand motions. 

For each validation code item, check the box in the “Validation” column if the response coded by the 
assessor is not what you have coded. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



As mentioned earlier, during the practice child assessment, the evaluator simultaneously 
coded the child’s response to the open-ended assessment items listed in the Assessment Certification 
Form. These open-ended items were flagged in the CAI program for quality control review and a screen 
showing how the trainee coded each of those answers to the questions was accessed at the end of the 
assessment for review with the evaluator as shown in exhibit 4-8. 
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Exhibit 4-8. Quality review screen: School year 2003-04 



Il2 LCL5-K Child Assessments OSlfxl 




INCORRECT 

25 . 2*45 

CORRECT 



In this secsrce. what does tte swd jammed r msf\? 
II 1KHHI 



A-:Q316Cttc^yPd 
ATQ.Vfl RdRttl 

ACQ335 RdVafaw 
ACQJWJ f/dUbe 

C4d ill/114 Hodftcd by njhs Ctrty CHUB 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



Exhibit 4-9 presents an example of a question from the Assessment Certification Form, 
which required the evaluator to evaluate the required assessment administration skills: reading verbatim, 
using appropriate gesturing, using the appropriate probe, and accurately coding the child’s response to 
this open-ended question. The evaluator checked the boxes only if the assessor did not demonstrate the 
correct behavior. For all questions observed by the evaluator and listed in the Assessment Certification 
Form, the evaluator recorded if the assessor did not read verbatim or gesture appropriately. Evaluators 
only marked a box when the assessor did not use the appropriate probe for those questions that displayed 
probes (a box labeled “Assessor Used Inappropriate Probe?” in the Assessment Certification Form). Also, 
the evaluators only marked the box if their response code did not match the assessors’ for open-ended 
questions (a box labeled “Validation Code Did Not Match” was presented in the Assessment Certification 
Form). 
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Exhibit 4-9. Sample question from Assessment Certification Form: School year 2003-04 





DID NOT 
READ 
QUESTION 
VERBATIM 


GESTURED 

INAPPRO- 

PRIATELY 


VALIDATION 
CODE DID 
NOT MATCH 


R5.245 What do you think is the difference between the 
storyteller’s bedroom and the sister’s bedroom?** 

IF STUDENT SAYS, “ A mess” OR “A lot of junk”, 
PROMPT: “How is it different?” 

CORRECT 1 

STORY TELLER’S BEDROOM IS: 

(BIGGER MESS/MORE 
MESS/MESSIER) 

(MORE JUNK/JUNKIER) 

(IS A MORE DISGUSTING SPOT) 

CORRECT ANSWER MUST INVOLVE 
A COMPARISON 

INCORRECT 2 

(STORY TELLER’S BEDROOM IS 

CLEAN/AS MESSY AS SISTER’S BEDROOM) 
DETAILS ABOUT THE SISTER’S 
BEDROOM 
EXAMPLES: 

(PIZZA UNDER THE DESK) 

(SOCKS ON THE LIGHT) 

(MESS/ALSO A MESS) 

(JUNK/ A LOT OF JUNK) 

REFUSED 777 

DON’T KNOW 999 

ASSESSOR USED INAPPROPRIATE PROBE? □ 


□ 


□ 


□ 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



At the end of the “live” child assessment, after the child had been escorted from the room, 
the evaluator and the trainee reviewed each trainee’s overall performance on the half of the cognitive 
assessment that he or she conducted. After discussing the ratings in section 1, the trainee accessed the 
quality control screen. The trainee and evaluator reviewed their codes for each open-ended question asked 
in section 2 of the Assessment Certification Form. 



The evaluator then scored the certification assessment using the scoring form shown in 
exhibit 4-10. The evaluator first counted the number of boxes checked in section 1: Rapport Building of 
the Assessment Certification Form and recorded that number in the appropriate row of Form A of the 
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Certification Scoring Form. The evaluator then counted the number of check marks for each section of the 
assessment that was observed and recorded those numbers in the appropriate boxes of Form A. Exhibit 4- 
1 1 presents an example of a completed Form A. In this example, the evaluator observed one trainee who 
administered Reading Routing, Reading Y ellow and Math Routing. The evaluator recorded zero (0) check 
marks for section I, zero (0) check marks for Reading Routing, one (1) check mark for Reading Yellow, 
and zero (0) check marks for Math Routing. The total check marks recorded was one (1). 

The evaluator then calculated the number of possible points for the sections of the 
assessment observed using the Total Possible Points Chart displayed in exhibit 4-12. This chart was 
necessary for the assessment certification because each trainee only conducted half of the cognitive 
assessment with a child respondent; this chart helped the evaluator use the correct denominator for 
calculating the score. For each section of the assessment observed, the evaluator circled the last question 
recorded in the Assessment Certification Form and wrote the number of possible points in the appropriate 
box in the second section of the scoring form (Form B) (see exhibit 4-13). Continuing with the example 
started earlier, the evaluator observed all of Reading Routing (6 possible points), all of Reading Yellow 
(21 possible points), and all of Math Routing (8 possible points). In addition, section 1 contributes 6 
possible points. Based on these sections of the assessment, the total number of possible points for this 
trainee is 4 1 . 



Finally, the evaluator determined the proportion correct by using the Proportion Correct 
Chart displayed in exhibit 4-14. Continuing the example, the evaluator found the column on the 
Proportion Correct Chart that displayed the total possible points for the trainee based on the portions of 
the assessment observed (41) and the row on the Proportion Correct Chart that displayed the total check 
marks recorded for the trainee (1), and recorded the proportion from the corresponding box on the 
Proportion Correct Chart in the box under Step 6 (exhibit 4-15) on the scoring form (.98). Once the 
evaluator completed the scoring, the trainee was rated as Certified, Remedial Action, or Failed. 
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Exhibit 4-10. Certification Scoring Form: School year 2003-04 



Certification Scoring 

Step 1: Record the number of check marks from Section 1: Rapport Building and Section 2: Specific 

Assessment Activities in the appropriate boxes of Form A. 

Step 2: Sum each row and record total in Row Totals column of Form A. 

Step 3: Sum Row Totals column (including number of check marks from Section 1) and record in Total 

check marks box of Foim A. 



Form A: Checks Marks Recorded in Sections 1 and 2 



Assessment 


Routing Form 


Red Form 


Yellow Form 


Blue Form 


Row Totals 


Reading 












Mathematics 












Science 












Number of check marks from Section 1: Rapport Building: 




Total Check Marks: 





Step 4: Using the Total Possible Points Chart, circle the last question in the assessment certification 

form for each routing and sub-domain that the trainee completed. Put the number to the right of 
the item in the appropriate box of Form B. For example, the Trainee completed Reading 
Routing and through item R5.245 of Reading Red, the total number of possible points is 20 (6 
for Routing and 14 for Red). Record 6 in the Reading row under Routing Form and record 14 
under Red Form and 20 in the Row Totals column of Form B. 

Step 5: Sum Row Totals column (including Possible Points for Section 1) and record in Total Possible 
Points box of Form B. 



Form B: Total Possible Points 



Assessment 


Routing Form 


Red Form 


Yellow Form 


Blue Form 


Row Totals 


Reading 












Mathematics 












Science 












Possible Points for Section 1: Rapport Building: 


6 


Total Possible Points: 





Step 6: Use Proportion Correct Chart to determine the proportion correct and write that proportion in 

this box: 

Result: If Proportion correct is greater than or equal to .85, Assessor is Certified. 

If Proportion correct is between .70 and .85, Assessor requires Remedial Action (Complete 
Remedial Task Chart below) 

If Proportion correct is less than .70, Assessor Failed. 

See note at end of exhibit. 
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Exhibit 4-10. Certification Scoring Form: School year 2003-04 — Continued 



Remedial Task Chart 

Circle the number of the task(s) needing improvement. Discuss remedial action with the assessor. 


TASK 


# 


REMEDIAL ACTION 


APPROPRIATE PACE 


1 


PRACTICE/REVIEW JOB AIDS 


AVOID COACHING 


2 


REVIEW THE JOB AIDS 


READ VERBATIM 


3 


PRACTICE READING FROM THE SCREEN 


GESTURING APPROPRIATELY 


4 


PRACTICE GESTURING 


USE APPROPRIATE PROBES 


5 


REVIEW THE JOB AIDS/Q-BY-Qs 


VALIDATION CODE DID NOT MATCH 


6 


REVIEW THE Q-BY-Qs 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



Exhibit 4-11. Example of Form A for Trainee 1 : School year 2003-04 



Form A: Checks Marks Recorded in Sections 1 and 2 



Assessment 


Routing Form 


Red Form 


Yellow Form 


Blue Form 


Row Totals 


Reading 


0 




1 




1 


Mathematics 


0 








0 


Science 












Number of check marks from section 1: Rapport Building: 


0 


Total Check Marks: 


1 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten 
Class of 1998-99 (ECLS-K), spring 2004. 
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Exhibit 4-12. Total Possible Points Chart: School year 2003-04 



Reading 



Total Possible Points Chart 



Routing 


Red 


Yellow 


Blue 


Item # 


Total 

Points 


Item # 


Total 

Points 


Item # 


Total 

Points 


Item # 


Total 

Points 


R5.055 


3 


R5.185 


4 


R5.270 


4 


R5.405 


3 


R5.060 


6 


R5.215 


7 


R5.285 


7 


R5.415 


6 




R5.235 


10 


R5.305 


10 


R5.460 


8 


R5.245 


14 


R5.335 


12 


R5.470 


11 


R5.250 


17 


R5.340 


15 


R5.480 


14 


R5.260 


20 


R5.365 


18 


R5.530 


17 






R5.375 


21 


R5.535 


20 



Mathematics 



Routing 


Red 


Yellow 


Blue 


Item # 


Total 


Item # 


Total 


Item # 


Total 


Item # 


Total 




Points 




Points 




Points 




Points 


M5.045 


3 


M5.125 


3 


M5.185 


2 


M5.275 


4 


M5.070 


5 


M5.135 


5 


M5.205 


6 


M5.285 


6 


M5.085 


8 


M5.155 


7 


M5.210 


9 


M5.325 


9 






M5.160 


9 


M5.235 


13 


M5.335 


12 






M5.170 


11 


M5.245 


15 


M5.340 


16 










M5.265 


18 


M5.360 


19 














M5.365 


23 



Science 



Routing 


Red 


Yellow 


Blue 


Item # 


Total 

Points 


Item # 


Total 

Points 


Item # 


Total 

Points 


Item # 


Total 

Points 


S5.015 


3 


S5.125 


3 


S5.205 


2 


S5.285 


4 


S5.020 


6 


S5.140 


6 


S5.215 


5 


S5.290 


6 


S5.040 


9 


S5.160 


8 


S5.225 


8 


S5.295 


10 


S5.085 


11 


S5.170 


11 


S5.230 


11 


S5.300 


12 




S5.180 


14 


S5.235 


13 


S5.310 


15 






S5.250 


15 


S5.325 


18 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, 
Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 
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Exhibit 4-13. Example of Form B: School year 2003-04 



Form B: Total Possible Points 



Assessment 


Routing Form 


Red Form 


Yellow Form 


Blue Form 


Row Totals 


Reading 


6 




21 




28 


Mathematics 


7 








7 


Science 












Possible Points for section 1: Rapport Building: 


6 


Total Possible Points: 


41 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, 
Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 
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Exhibit 4-14. Proportion Correct Chart: School year 2003-04 





69 


70 




73 


74 


76 


77 


78 


79 


80 


81 


82 


83 


84 


85 


86 


87 


88 


89 


90 


92 


93 


1 


0.99 


0.99 




0.99 


0.99 


0.99 


0.99 


0.99 


0.99 


0.99 


0.99 


0.99 


0.99 


0.99 


0.99 


0.99 


0.99 


0.99 


0.99 


0.99 


0.99 


0.99 


2 


0.97 


0.97 


0.97 


0.97 


0.97 


0.97 


0.97 


0.97 


0.97 


0.98 


0.98 


0.98 


0.98 


0.98 


0.98 


0.98 


0.98 


0.98 


0.98 


0.98 


0.98 


0.98 


3 


0.96 


0.96 


0.96 


0.96 


0.96 


0.96 


0.96 


0.96 


0.96 


0.96 


0.96 


0.96 


0.96 


0.96 


0.96 


0.97 


0.97 


0.97 


0.97 


0.97 


0.97 


0.97 


4 


0.94 


0.94 


0.94 


0.95 


0.95 


0.95 


0.95 


0.95 


0.95 


0.95 


0.95 


0.95 


0.95 


0.95 


0.95 


0.95 


0.95 


0.95 


0.96 


0.96 


0.96 


0.96 


5 


0.93 


0.93 


0.93 


0.93 


0.93 


0.93 


0.94 


0.94 


0.94 


0.94 


0.94 


0.94 


0.94 


0.94 


0.94 


0.94 


0.94 


0.94 


0.94 


0.94 


0.95 


0.95 


6 


0.91 


0.91 


0.92 


0.92 


0.92 


0.92 


0.92 


0.92 


0.92 


0.93 


0.93 


0.93 


0.93 


0.93 


0.93 


0.93 


0.93 


0.93 


0.93 


0.93 


0.93 


0.94 


7 


0.90 


0.90 


0.90 


0.90 


0.91 


0.91 


0.91 


0.91 


0.91 


0.91 


0.91 


0.91 


0.92 


0.92 


0.92 


0.92 


0.92 


0.92 


0.92 


0.92 


0.92 


0.92 


8 


0.88 


0.89 


0.89 


0.89 


0.89 


0.89 


0.90 


0.90 


0.90 


0.90 


0.90 


0.90 


0.90 


0.90 


0.91 


0.91 


0.91 


0.91 


0.91 


0.91 


0.91 


0.91 


9 


0.87 


0.87 


0.88 


0.88 


0.88 


0.88 


0.88 


0.88 


0.89 


0.89 


0.89 


0.89 


0.89 


0.89 


0.89 


0.90 


0.90 


0.90 


0.90 


0.90 


0.90 


0.90 


10 


0.86 


0.86 


0.86 


0.86 


0.86 


0.87 


0.87 


0.87 


0.87 


0.88 


0.88 


0.88 


0.88 


0.88 


0.88 


0.88 


0.89 


0.89 


0.89 


0.89 


0.89 


0.89 


11 


0.84 


0.84 


0.85 


0.85 


0.85 


0.86 


0.86 


0.86 


0.86 


0.86 


0.86 


0.87 


0.87 


0.87 


0.87 


0.87 


0.87 


0.88 


0.88 


0.88 


0.88 


0.88 


12 


0.83 


0.83 


0.83 


0.84 


0.84 


0.84 


0.84 


0.85 


0.85 


0.85 


0.85 


0.85 


0.86 


0.86 


0.86 


0.86 


0.86 


0.86 


0.87 


0.87 


0.87 


0.87 


13 


0.81 


0.81 


0.82 


0.82 


0.82 


0.83 


0.83 


0.83 


0.84 


0.84 


0.84 


0.84 


0.84 


0.85 


0.85 


0.85 


0.85 


0.85 


0.85 


0.86 


0.86 


0.86 


14 


0.80 


0.80 


0.81 


0.81 


0.81 


0.82 


0.82 


0.82 


0.82 


0.83 


0.83 


0.83 


0.83 


0.83 


0.84 


0.84 


0.84 


0.84 


0.84 


0.84 


0.85 


0.85 


15 


0.78 


0.79 


0.79 


0.79 


0.80 


0.80 


0.81 


0.81 


0.81 


0.81 


0.81 


0.82 


0.82 


0.82 


0.82 


0.83 


0.83 


0.83 


0.83 


0.83 


0.84 


0.84 


16 


0.77 


0.77 


0.78 


0.78 


0.78 


0.79 


0.79 


0.79 


0.80 


0.80 


0.80 


0.80 


0.81 


0.81 


0.81 


0.81 


0.82 


0.82 


0.82 


0.82 


0.83 


0.83 


17 


0.75 


0.76 


0.76 


0.77 


0.77 


0.78 


0.78 


0.78 


0.78 


0.79 


0.79 


0.79 


0.80 


0.80 


0.80 


0.80 


0.80 


0.81 


0.81 


0.81 


0.82 


0.82 


18 


0.74 


0.74 


0.75 


0.75 


0.76 


0.76 


0.77 


0.77 


0.77 


0.78 


0.78 


0.78 


0.78 


0.79 


0.79 


0.79 


0.79 


0.80 


0.80 


0.80 


0.80 


0.81 


19 


0.72 


0.73 


0.74 


0.74 


0.74 


0.75 


0.75 


0.76 


0.76 


0.76 


0.77 


0.77 


0.77 


0.77 


0.78 


0.78 


0.78 


0.78 


0.79 


0.79 


0.79 


0.80 


20 


0.71 


0.71 


0.72 


0.73 


0.73 


0.74 


0.74 


0.74 


0.75 


0.75 


0.75 


0.76 


0.76 


0.76 


0.76 


0.77 


0.77 


0.77 


0.78 


0.78 


0.78 


0.78 


21 


0.70 


0.70 


0.71 


0.71 


0.72 


0.72 


0.73 


0.73 


0.73 


0.74 


0.74 


0.74 


0.75 


0.75 


0.75 


0.76 


0.76 


0.76 


0.76 


0.77 


0.77 


0.77 


22 


0.68 


0.69 


0.69 


0.70 


0.70 


0.71 


0.71 


0.72 


0.72 


0.73 


0.73 


0.73 


0.73 


0.74 


0.74 


0.74 


0.75 


0.75 


0.75 


0.76 


0.76 


0.76 


23 


0.67 


0.67 


0.68 


0.68 


0.69 


0.70 


0.70 


0.71 


0.71 


0.71 


0.72 


0.72 


0.72 


0.73 


0.73 


0.73 


0.74 


0.74 


0.74 


0.74 


0.75 


0.75 


24 


0.65 


0.66 


0.67 


0.67 


0.68 


0.68 


0.69 


0.69 


0.70 


0.70 


0.70 


0.71 


0.71 


0.71 


0.72 


0.72 


0.72 


0.73 


0.73 


0.73 


0.74 


0.74 


25 


0.64 


0.64 


0.65 


0.66 


0.66 


0.67 


0.68 


0.68 


0.68 


0.69 


0.69 


0.70 


0.70 


0.70 


0.71 


0.71 


0.71 


0.72 


0.72 


0.72 


0.73 


0.73 


26 


0.62 


0.63 


0.64 


0.64 


0.65 


0.66 


0.66 


0.67 


0.67 


0.68 


0.68 


0.68 


0.69 


0.69 


0.69 


0.70 


0.70 


0.70 


0.71 


0.71 


0.72 


0.72 


27 


0.61 


0.61 


0.63 


0.63 


0.64 


0.64 


0.65 


0.65 


0.66 


0.66 


0.67 


0.67 


0.67 


0.68 


0.68 


0.69 


0.69 


0.69 


0.70 


0.70 


0.71 


0.71 


28 


0.59 


0.60 


0.61 


0.62 


0.62 


0.63 


0.64 


0.64 


0.65 


0.65 


0.65 


0.66 


0.66 


0.67 


0.67 


0.67 


0.68 


0.68 


0.69 


0.69 


0.70 


0.70 


29 


0.58 


0.59 


0.60 


0.60 


0.61 


0.62 


0.62 


0.63 


0.63 


0.64 


0.64 


0.65 


0.65 


0.65 


0.66 


0.66 


0.67 


0.67 


0.67 


0.68 


0.68 


0.69 


30 


0.57 


0.57 


0.58 


0.59 


0.59 


0.61 


0.61 


0.62 


0.62 


0.63 


0.63 


0.63 


0.64 


0.64 


0.65 


0.65 


0.66 


0.66 


0.66 


0.67 


0.67 


0.68 



NOTE: Proportion correct = (Total Possible Points - Number of Check Marks)/Total Possible Points. EXAMPLE: If an assessor received 3 check marks out of a total of 24 possible points, the 
proportion correct is .88. This Assessor would be certified. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 




Exhibit 4-15. Example of Proportion Correct Chart: School year 2003-04 



Step 6: Use Proportion Correct Chart to determine the proportion correct and write that proportion in 

this box: [ ~ j 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



Table 4-1 presents the results of the training certification. There were 242 assessors and 81 
field supervisors for a total of 323 trainees who were certified; 16 assessors were only trained to complete 
parent interviews. Trainees scoring 85 percent or above were certified as qualified to administer the child 
assessments. Trainees who scored between 70 and 84 percent were required to complete remedial training 
and have an additional certification conducted before beginning assessments. 



Table 4-1. Results of child assessment certification, fifth grade: School year 2003-04 



Trainees 


Number 


Percent 


Total 


323 


100 


Score on certification form 


85 percent or above 


320 


99.0 


70-84 percent 


3 


1.0 


Below 70 percent 


0 


0 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



The majority of the trainees (99.0 percent) scored above 85 percent on the certification form, 
with only 1.0 percent (n=3) scoring between 70 and 84 percent. None of the trainees failed to meet the 
70 percent threshold on the Assessment Certification Form. All trainees who needed remedial training 
were certified qualified to administer the child assessments after they conducted a second assessment on a 
fifth grade-aged child who was not part of the ECLS-K sample. 

The three trainees who required remedial action were assigned remedial training, exercises, 
and practice assessments to perform on an age-appropriate child in their area. Their supervisor observed 
the practice assessment and certified them. The scoring form specifies the type of remedial training 
activities that were assigned based on the type of problems observed by the evaluator (see exhibit 4-16). 
Once the remedial training activities were completed, the field supervisor or field manager was 
responsible for conducting an additional certification assessment with the trainee with a fifth grade-aged 
child not in the ECLS-K sample prior to being able to start work. 
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Exhibit 4-16. Certification Scoring Form: School year 2003-04 



Remedial Task Chart 

Circle the number of the task(s) needing improvement. Discuss remedial action with the assessor. 


TASK 


# 


REMEDIAL ACTION 


APPROPRIATE PACE 


1 


PRACTICE/REVIEW JOB AIDS 


AVOID COACHING 


2 


REVIEW THE JOB AIDS 


READ VERBATIM 


3 


PRACTICE READING FROM THE SCREEN 


GESTURING APPROPRIATELY 


4 


PRACTICE GESTURING 


USE APPROPRIATE PROBES 


5 


REVIEW THE JOB AIDS/Q-BY-Qs 


VALIDATION CODE DID NOT MATCH 


6 


REVIEW THE Q-BY-Qs 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



4.3.2.3 Parent Interview Training and Certification 

Training assessors to conduct the parent interview also included interactive lectures and role 
plays that were designed not only to review the intent of the questions but also to demonstrate the 
different ways that parents may answer questions. Ambiguous answers were included in both training 
interactive lectures and role-play scripts to provide practice in probing and handling a variety of 
responses. Parent interview QxQs were carefully reviewed throughout training so that 
assessors/interviewers would be prepared for a variety of responses from parents. The culmination of 
parent interviewer training was a final certification role play designed to test all the protocols and 
techniques reviewed during training. The final role play was conducted using a scripted parent interview 
and the final parent role play evaluation form. The final role-play script was designed to test the trainee’s 
understanding of the interview content, proper interviewing techniques, including probing, and accurate 
response recording in CATI. Trainees were paired up and each completed half of a parent interview as the 
interviewer. As they conducted the interview as the respondent, trainees completed the evaluation form 
that consisted of a checklist to evaluate key areas, such as contacting and selecting the respondent, asking 
the questions verbatim, probing properly, and following the correct question path. Two points were given 
for each item performed correctly. Trainees had to score at least 30 out of a possible 36 points to pass. 

Table 4-2 presents the results of the parent certification. 
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Table 4-2. Results of parent interview certification, fifth grade: School year 2003-04 



Trainees 


Number 


Percent 


Total 


308 


100 


Score on certification form 


85 percent or above 


294 


95.5 


70-84 percent 


14 


4.5 


Below 70 percent 


0 


0 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



All trainees who expected to conduct parent interviews completed the parent certification 
procedures. The majority of trainees (95.5 percent or 294 trainees) were successfully certified on the 
parent interview. Parent certification scores were not reported for sixteen trainees. The 14 trainees who 
did not pass the parent certification at training worked with their supervisors to improve the areas in 
which they lacked skills. These staff completed another parent certification role play with their 
supervisor. All 14 staff were certified on the parent interview after the remedial training. 



4.4 Fall Preassessment School Contact 

Beginning in September 2003, all participating ECLS-K schools (i.e., schools that 
participated in previous rounds of data collection) were contacted by telephone to prepare for the spring 
data collection. When children were identified as having transferred to another school, the child’s new 
school (and district, if necessary) was recruited. As noted in section 4.3.1, the advance contact 
served several purposes. It reminded schools about the study and reacquainted them with the study’s 
procedures; it provided supervisors the opportunity to persuade the schools to participate; and it allowed 
the collection of information necessary for the spring 2004 national data collection. There were four 
primary tasks to be accomplished during the fall contact. These were to schedule appointments to conduct 
the child assessments in the spring; to verify parent consent procedures; to identify the children’s 
teachers; and to identify children who had withdrawn from the school and obtain locating information 
about their new schools. The fall contact activities are described below. 
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4.4.1 



Advance Mailings 



In September 2003, an advance package was mailed via Federal Express to all participating 
ECLS-K schools, i.e., schools that participated in third grade, asking them to prepare for the 
preassessment contact telephone call. The schools were asked to identify a school staff coordinator to 
serve as a liaison with the study (in original sampled schools, this person was usually the coordinator 
from the previous rounds of data collection). A package containing study materials was sent to the 
schools. The package contained the following materials and was customized by school type — original or 
transfer schools: 



Original school advance package: 



■ A letter printed on ECLS-K letterhead reminding school staff about the study, 
describing the fifth-grade data collection, and alerting the school coordinator of the 
advance contact in the fall; 

■ A School Summary Sheet (original schools) — a two-page document providing a brief 
review of the study to date and the fifth-grade data collection activities; 

■ A Study Findings Sheet (original schools) — a summary of findings about children 
from the previous rounds of data collection; and 

■ An ECLS-K Study Children Form and Instructions — a listing of all the sampled 
children and instructions for completing the form with specific information such as 
the children’s continued attendance at the school, their grade, their teachers’ names, 
classroom numbers, receipt of special education services, and receipt of assessment 
accommodations/exclusions before the preassessment call. 



Transfer schools: 



■ A letter printed on ECLS-K letterhead introducing school staff to the study, describing 
the fifth-grade data collection, and alerting the school coordinator of the advance 
contact in the fall; 

■ A School Summary Sheet (transfer schools) — a two-page document providing a brief 
overview of the study and the fifth-grade data collection activities; 

■ A Study Findings Sheet — a summary of findings about children from the previous 
rounds of data collection; and 

■ An ECLS-K Study Children Form and Instructions — a listing of all the sampled 
children and instructions for completing the form with specific information such as 
the children’s continued attendance at the school, their grade, their teachers’ names, 
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classroom numbers, receipt of special education services, and receipt of assessment 
accommodations/exclusions before the preassessment call. 



4,4.2 Fall Preassessment School Coordinator Contact 

The preassessment contacts were made by telephone from September through December 
2003. The preassessment school contacts were successful in meeting all four tasks described above. 
Contacting original sampled schools to set up the spring assessment and identifying children who 
withdrew from their spring-third grade school and moved into their fifth-grade transfer school, enabled 
the identification of schools that were ineligible for fifth-grade data collection. Schools were determined 
to be ineligible for fifth-grade data collection if no ECLS-K sampled children were currently enrolled. 
Original sampled schools became ineligible because fourth grade was the highest grade in the school or 
because the school had closed, that is, was no longer operational. More transfer schools were determined 
to be ineligible as children transferred out of them into other schools. 

During the preassessment contact, the field supervisor contacted the school coordinator to 
collect some basic information about the school and some detailed information about each ECLS-K 
sampled child. The field supervisor used the School Information Form to collect basic information about 
the school, such as school start and end dates, vacation and holiday schedules, and parking directions. The 
form was also used to determine if the school was a year-round school, taught fifth grade, or required new 
parent consent, and to obtain information on class organization. The supervisor used the Child Work Grid 
(exhibit 4-17) to collect basic information about the child such as his or her grade, the name and 
classroom number of the child’s primary teacher to link the child to a teacher, and whether the child had 
an Individualized Education Program (IEP) or its equivalent. In addition, in original sample schools, the 
assessment date was scheduled; assessment dates for new transfer schools were scheduled in the spring. 

Reviewing parent consent with the school. Because parental consent was obtained in the 
base year and re-obtained in the third-grade year, field supervisors did not raise parental consent with the 
school coordinator unless the school district required it. However, if the school was a transfer school then 
the supervisor did ask the school coordinator whether it was necessary to obtain parental consent. If the 
schools required current consent forms or changed the type of consent that was required (e.g., from 
implicit to explicit), parent letters and consent forms were either mailed to the school for distribution to 
parents or sent directly to parents by Westat depending on the schools’ preference. Parent cover letters 
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Exhibit 4-17. Child Work Grid, spring-fifth grade: School year 2003-04 



SUPERVISOR VERSION - BLANK 

R6 Region: R6 Workarea: 



Round 6 ECLS-K Child Work Grid 



School ED: 



School name: 



Year round school? 



Page of 



1 


2 


3 


4 


5 


6 


7 


8 


9 








Child Here? screen 


Child Locating screen 


Child Here? 
screen 


Child Work 
Grid screen 


Child Work 
Grid screen 


Teacher Info/Link 
Child screen 


Child Work Grid 
screen 


Continue 
with column 
12 on the 
back of this 
page for 
each child 
listed. 


Student 

name/ID 


Follow 

status 


Is this child still 
attending this school? 

If no-complete column 
4 and go to the next 
child listed 

If yes-skip to column 5 


What school did the child transfer to? 

(name/address/phone) 


What grade 
is this child 
in? 


Does this child 
have an 
IEP or its 
equivalent? 

If no: skip to 
column 9 
If ves: comnlete 
columns 7, 8 
and 9. 


Does the IEP 
specify that 
the child 
cannot take 
part in 
standardized 
assessments? 


What is the name 
and classroom 
number of this 
child’s special 
education 
teacher? 


Does the child 
require any of the 
accommodations or 
exclusions listed on 
the Instruction 
Sheet (page 2)? 

If yes, which ones? 















































































































See note at end of exhibit. 
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Exhibit 4-17. Child Work Grid, spring-fifth grade: School year 2003-04 — Continued 





Respond for the Math teacher if column 1 1 = M. 
Respond for the Science teacher if column 1 1 = S 


10 


11 


12 


13 


14 


IS 


16 


17 






Teacher Info/Link Child 
screen 


Teacher 
Info/Link Child 
screen 


Teacher 
Info/Link Child 
screen 


Teacher Info/Link Child 
screen 


Teacher 
Info/Link Child 
screen 


Teacher 
Info/Link Child 
screen 


Student 

name/ID 


Domain 

Flag 

S=Science 

M=Math 


What is the first and last name 
of this child’s Reading 
teacher? 


What time does 
this child’s 
Reading class 
meet? 
Record as 
XX:XX am/pm 


What is the 
classroom 
number of this 
child’s Reading 
class? 


What is the first and last name 
of this child’s Math OR 
Science teacher? 


What time does 
this child’s Math 
OR Science class 
meet? 
Record as 
XX:XX am/pm 


What is the 
classroom 
number of this 
child’s Math OR 
Science class? 



































































































SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 




and consent forms were available in English and Spanish. Overall, 16 percent of all schools contacted 
required an updated parental consent. Of the schools requiring updated parental consent, approximately 
50 percent required explicit consent from the parents. 

Collecting information about ECLS-K sampled children. Field supervisors primarily used 
the Child Work Grid to collect information from the school coordinator about the ECLS-K sampled 
children. The school coordinator was asked to complete the school version of the form to expedite the 
call. Field supervisors collected information from the school coordinators about the ECLS-K sampled 
children still enrolled in the school, including the child’s current grade, the name and classroom for the 
child’s regular teacher in order to link the child to a teacher, and whether or not the child had an 
Individualized Education Program (IEP) (see exhibit 4-18). If the child had an IEP, then the name and 
classroom of the child’s special education teacher was noted, along with whether the child required any 
accommodations to participate in the direct cognitive assessment. Through this process, 1,088 children 
were identified with IEPs and 793 special education teachers were li nk ed to these children. The 
accommodations to the fifth-grade direct cognitive assessment were the same as those for the third-grade 
direct cognitive assessment (see section 4.5.2). Field supervisors contacted the teachers of the ECLS-K 
children as necessary for any of this information. Field supervisors also identified the respondent for the 
student record abstract for each child. 

If a child was identified as having transferred out of the school, the field supervisor asked the 
school coordinator to provide the names, addresses, and telephone numbers of these transfer schools Of 
those children who transferred, only a subset was followed to their new school (see section 3.6.1 in 
chapter 3 for more detail on how mover children were subsampled). If the new school belonged to a 
district that was new to the study, the district was mailed a letter with the new school name. A field 
supervisor contacted the district by telephone and recruited the district into the study before any contact 
was made with the school. If the district was already cooperating, the district was notified by mail and the 
new school was contacted and recruited directly. Field supervisors also verified with the school that no 
child who had previously transferred had returned to the school. 



4-38 




Exhibit 4-18. Individualized Education Program (IEP) process, spring fifth-grade: School year 2003-04 



WHO WILL BE ASSESSED? 



DOES CHILD HAVE IEP OR 1FSP ON FILE? 



' Does IEP specify that child cannot 
i participate in standardized 

assessments? A 



No 

Use Standard Assessment 
Procedures 



Yes 

Exclude child from 



f No N 

Does child require an 
s assessment accommodation?^ 



Link child to a regular 
and special education 
teacher 



^ Yes \ 

Is the required assessment 
accommodation olFered by 
V ECLS-K? A 



^ No ^ 

Use standard assessment 
procedures 



/ Yes \ 

Assessment conducted with ’ 
provision of accommodation. 

Is the accommodation an IEP? . 



' No 
Exclude child from 



Link child to a 
regular and special 
k education teacher , 



Yes ^ 
Link child to a regular and 
special education 
V teacher 



(Child is linked to a regular! 
V teacher only 











Contacting families of homeschooled children. As part of the fall preassessment contact, 
children in the ECLS-K sample who were homeschooled in previous rounds were identified. The status of 
home-schooled children who were identified in rounds 1 through 4 was verified with their parents and 
updated as necessary. In addition, during the preassesment contact some schools identified homeschooled 
children. Their status was also verified with their parents during data collection. Parents of these children 
were contacted by telephone in September through December 2003 to determine if the child was still 
homeschooled or had enrolled in a school. If the child had enrolled in a school, the new school was 
contacted and recruited into the study. Parents of children who were still schooled at home were notified 
about the next round of data collection in the spring. 

Identifying the key child in classrooms with multiple study children. In grade 5, the 
design of the child-level teacher questionnaire was changed to include collecting data about the child’s 
reading class and mathematics or science class. In previous rounds, children were taught primarily in self- 
contained classrooms and teachers only reported classroom level information once for the classroom. Due 
to the grade 5 design change, the teacher-child links were restructured to include the domain (reading, 
mathematics, or science) and a classroom identifier (e.g., room number and the time the class meets). A 
unique link number was created for each teacher/child/domain/class combination. In order to reduce data 
collection burden for teachers who were linked to multiple sample children in the same class, a “Key 
Domain Child” was identified for each separate subject and class that each teacher taught. The teachers 
were asked to report classroom level information only once in the questionnaire for the key domain child 
and child-level information for all sampled children in their class. Field supervisors collected the teacher- 
child-domain-classroom link information about each child and entered the information into the FMS. The 
information was used to generate the hardcopy teacher questionnaires (see section 5.4.3 for more 
information). 



4.4.3 Fall Preassessment Contact Results 

The goals for the fall preassessment contact with schools were the following: (1) set 
appointments for the spring assessment in original sample schools, (2) identify schools that ended at 
fourth grade to determine the school to which the sample children transferred, (3) identify children who 
changed schools since third grade, (4) link children to teachers and domains (i.e., reading, mathematics, 
science) for the advance school and teacher questionnaire mailings, and (5) contact as many transfer 
schools as possible within the field period to ascertain whether the child was still there and recruit the 
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school into the study. It was not expected that every transfer school identified within the fall contact could 
be contacted within the fall field period because of the numbers of children that were expected to move. It 
was also expected that additional schools would be contacted during the spring round because children 
were expected to continue to move between fall and spring of the school year. 



Approximately 30 percent of the ECLS-K sampled children transferred to other schools 
between the spring of third grade and beginning of fifth grade (i.e., fall 2003). At the start of the fall 2003 
field period, there were 1,016 transfer schools identified prior to the fall contact. An additional 744 
transfer schools were identified during the fall contact field period (September-December) for a total of 
1,760 transfer schools at the end of the field period. All of the transfer schools identified during the fall 
were contacted at least once during the field period to ascertain if the child was still enrolled. During the 
fall preassessment contact, 4,039 children were identified as movers and processed in preparation for the 
spring data collection. Table 4-3 presents the status of these 4,039 movers after the fall preassessment 
contact. 



Table 4-3. Status of movers identified during fall preassessment contact: School year 2003-04 



Status 


Total 


Percent 


Total 


4,039 


100 


Fielded for assessment in spring 


2,381 


58.9 


Assess in school 


2,261 


55.9 


Assess in home 


120 


2.9 


Unlocatable 


279 


6.9 


Moved to nonsampled PSU 


128 


3.2 


Moved outside of U.S. 


27 


<1 


Subsample not followed 


1,220 


30.0 


End of field period 


4 


<1 



NOTE: Movers who had a status of “Fielded for assessment in spring” were treated like the nonmover cases during the spring data collection. 
Those with a status of “Unlocatable” could not be located during the fall preassessment contact; additional attempts to locate them continued in 
spring data collection. Cases identified as “Moved to nonsampled PSU” or “Moved to outside of U.S.” were not fielded for the spring data 
collection because they had moved out of the designated data collection area. Cases with a status of “Subsample not followed” were not fielded in 
the spring because the sampling plan called for collecting data from only a subsample of movers (see chapter 3, section 3.6.1). These cases were 
not in the subsample to be followed. Only 4 cases with a status of “End of field period” were not contacted in the fall because the schools were 
either new to the school sample frame with limited information available to contact them or in school districts new to the sample that required an 
additional contact at the district level before the schools could be contacted for spring data collection. These cases were closed out and rolled over 
for contact in the spring. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 
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The fall preassessment contact protocol was completed for 100 percent of the original 
ECLS-K schools and 97 percent of the transfer schools (both those identified before and during the fall 
field period) within the fall field period. Twenty percent of schools were identified as out of scope, since 
they did not contain any sampled students (9 percent of original sampled schools and 25 percent of 
transfer schools). All of the schools that children transferred to as a result of the school ending at fourth 
grade, closing, or merging with another school were identified within the field period. Tables 4-4 through 
4-6 present the production reports for the fall preassessment contact for original sample and transfer 
schools. 



Table 4-4. Fall preassessment contact productivity report, by week: School 
year 2003-04 



Week 


Week ending 


Total schools completed 


Original 


Transfer 


1 


9/10/03 


9 


6 


3 


2 


9/17/03 


103 


78 


25 


3 


9/24/03 


284 


211 


73 


4 


10/11/03 


511 


360 


151 


5 


10/8/03 


766 


510 


256 


6 


10/15/03 


1001 


650 


351 


7 


10/22/03 


1233 


737 


496 


8 


10/29/03 


1463 


800 


663 


9 


1 1/5/03 


1637 


842 


795 


10 


11/12/03 


1826 


867 


959 


11 


11/19/03 


2047 


908 


1139 


12 


1 1/26/03 


2171 


927 


1244 


13 


12/3/03 


2240 


933 


1307 


14 


12/10/03 


2333 


940 


1393 


15 


12/17/03 


2392 


949 


1443 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood 
Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 
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Table 4-5. Fall preassessment contact production report — original schools: School year 2003-04 



Week # 


Week 

ending 


Response 

rate 

(percent) 


Total 

number 


Out of 
scope 


In scope 


No 

action 


Pending 


Complete 


Refusal 


Not 
contacted 
Final within field 
refusal period 


1 


9/10/03 


0.6 


967 


3 


964 


948 


9 


6 


0 


1 


0 


2 


9/1 1/03 


8.1 


993 


31 


962 


635 


247 


78 


1 


1 


0 


3 


9/24/03 


22.2 


1,004 


55 


949 


403 


331 


211 


2 


2 


0 


4 


10/0/03 


38.0 


1,016 


69 


947 


218 


634 


360 


3 


2 


0 


5 


10/8/03 


54.2 


1,026 


85 


941 


99 


328 


510 


2 


2 


0 


6 


10/15/03 


67.8 


1,048 


90 


958 


46 


256 


650 


3 


3 


0 


7 


10/22/03 


77.4 


1,044 


92 


952 


22 


187 


737 


3 


3 


0 


8 


10/29/03 


83.9 


1,048 


94 


954 


13 


135 


800 


3 


3 


0 


9 


1 1/5/03 


88.2 


1,050 


95 


955 


8 


99 


842 


3 


3 


0 


10 


11/12/03 


90.7 


1,051 


95 


956 


5 


78 


867 


3 


3 


0 


11 


11/19/03 


94.9 


1,051 


95 


957 


2 


41 


908 


3 


3 


0 


12 


1 1/26/03 


96.8 


1,052 


94 


958 


2 


22 


927 


2 


5 


0 


13 


12/3/03 


97.4 


1,052 


94 


958 


2 


16 


933 


1 


6 


0 


14 


12/10/03 


98.1 


1,052 


94 


958 


0 


9 


940 


0 


9 


0 


15 


12/17/03 


99.1 


1,052 


94 


958 


0 


0 


949 


0 


9 


0 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 
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Table 4-6. Fall preassessment contact production report — transfer schools, by week: School year 2003-04 



Week # 


Week 

ending 


Response 

rate 

(percent) 


Total 

number 


Out of 
scope 


In scope 


No action 


Pending 


Complete 


Refusal 


Final 

refusal 


Not 

contacted 

within 

field 

period 


1 


9/10/03 


0.3 


1,166 


2 


1,164 


1,160 


1 


3 


0 


0 


0 


2 


9/1 1/03 


2.1 


1,223 


25 


1,198 


1,133 


40 


25 


0 


0 


0 


3 


9/24/03 


5.9 


1,293 


57 


1,236 


1,096 


65 


73 


1 


1 


0 


4 


10/0/03 


11.5 


1,407 


92 


1,315 


1,071 


91 


1,511 


1 


1 


0 


5 


10/8/03 


18.5 


1,537 


155 


1,382 


997 


127 


256 


1 


1 


0 


6 


10/15/03 


25.3 


1,606 


218 


1,388 


873 


156 


351 


4 


4 


0 


7 


10/22/03 


3409 


1,708 


288 


1,420 


710 


201 


496 


7 


6 


0 


8 


10/29/03 


46.5 


1,785 


359 


1,426 


451 


299 


663 


8 


5 


0 


9 


1 1/5/03 


54.8 


1,847 


396 


1,451 


346 


292 


795 


10 


8 


0 


10 


11/12/03 


65.7 


1,898 


438 


1,460 


214 


266 


959 


11 


10 


0 


11 


11/19/03 


78.1 


1,922 


463 


1,459 


106 


191 


1,139 


12 


11 


0 


12 


1 1/26/03 


84.2 


1,957 


480 


1,477 


70 


142 


1,244 


6 


15 


0 


13 


12/3/03 


88.4 


1,972 


494 


1,478 


41 


108 


1,307 


4 


18 


0 


14 


12/10/03 


94.1 


1,988 


508 


1,480 


19 


42 


1,393 


3 


23 


0 


15 


12/17/03 


97.4 


1,997 


516 


1,481 


0 


0 


1,443 


0 


22 


16 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 
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Tracing Activities during Fifth-Grade Data Collection 



In June 2003 an attempt was made to obtain a current address for all eligible households 
using Telematch, a company that searches for current address information through its many databases. 
Telematch provided updated information, either a new or partial address, a new or partial telephone 
number, or both, for over half the households. For the remaining households. Telematch provided no new 
information beyond what was already contained Westat’s tracing and locating files. Table 4-7 presents the 
results of the spring Telematch search. 



Table 4-7. Tracing results for spring 2003 



Result 


Number 


Percent 


Total households searched 


13,390 


100.0 


Households located and entered 


7,641 


57.4 


into tracing database 






No new information provided 


5,749 


42.9 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



In order to ensure that as many of the sampled children as possible were contacted in the 
spring, locating efforts were conducted from June through early August 2003. Staff in Westat’s 
Telephone Research Center (TRC) traced the 829 children who could not be located during previous 
rounds of data collection. TRC staff used the Internet, telephone directories, and other means to locate 
these children and their households. When children and/or households were found, the new school and 
contacting information was entered into the tracing system database for fielding in the spring. Table 4-8 
presents the results of the summer tracing effort. 



Table 4-8. Results of the Telephone Research Center’s locating efforts for cases with a status of 
unbeatable at the beginning of fifth grade: School year 2003-04 



Result 


Number 


Percent 


Total cases worked 


827 


100.0 


Located and entered into database 


305 


36.8 


Unlocatable 


519 


62.6 


Final refusal 


5 


.06 



NOTE: “Unlocatable” means that the children and their households could not be found using the available tracing and locating strategies; “final 
refusal” means that the child’s family indicated that they did not want to participate. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 
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In mid-September 2003, all eligible households were mailed a letter asking the head of the 
household to record their current address and current school information for the sampled child on an 
enclosed postcard. In mid-October staff in Westat’s Telephone Research Center (TRC) began calling 
households that had not returned the postcard. By the end of December 2003, over 75 percent of the 
households had responded. Table 4-9 presents the results of the fall tracing contacts. 



Table 4-9. Results of the fall household tracing contact: School year 2003-04 



Result 


Total 


Percent 


Total households contacted 


13,379 


100.0 


New locating information provided 


2,457 


18.4 


No new locating information 


7,612 


56.9 


provided 






Refused 


83 


0.6 


No response to contact 


3,227 


24.1 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, 
Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 



4.5 Spring-Fifth Grade Data Collection 

All children who were assessed during the base year or for whom a parent interview was 
completed in the base year were eligible to be assessed in the spring-fifth grade data collection, with four 
exceptions: They were (1) children who became ineligible in an earlier round (because they died or 
moved out of the country), (2) children who were subsampled out in previous rounds because they moved 
out of the original schools and were not subsampled to be followed, (3) children whose parents 
emphatically refused to cooperate (hard refusals) in any of the data collection rounds since spring- 
kindergarten, and (4) children in the fifth-grade sample for whom there were neither first-grade nor third- 
grade data. Eligibility for the study was not dependent on the child’s current grade, that is, children were 
eligible whether they were promoted to fifth grade or were retained in fourth grade. In spring-fifth grade, 
the children attended 2,008 public schools and 356 private schools. Two-thirds of the public schools 
(1,355) and almost half of the private schools (166) were transfer schools. 

As in previous rounds of data collection, the field staff were organized into work areas, each 
with a data collection team consisting of one field supervisor and two or more assessors. The data 
collection teams were responsible for all data collection activities in their work areas; they conducted the 



4-46 



direct child assessments and the parent interviews, collected all school and teacher questionnaire and 
completed checklists. The majority of field staff members in fifth grade were continuing from previous 
rounds of data collection; a few new staff were hired in areas where no experienced ECLS-K staff lived. 



4.5.1 Spring Preassessment Activities 

Based on the information collected in the fall of 2003, packets of hard-copy teacher and 
school administrator questionnaires were assembled and mailed to schools in January and February 2004, 
along with letters confirming the scheduled visits to the school. Teachers and school administrators were 
asked to complete the questionnaires for pickup by the field supervisor or lead assessor on assessment 
day. Letters were also mailed to parents at this time reminding them of the spring-fifth grade data 
collection activities. 

Most preassessment contact with schools was by telephone starting in March 2004. Field 
supervisors contacted the school to confirm the assessment date and the receipt of the hard-copy 
questionnaires and to arrange for space to conduct the assessments. This follow-up call to the schools was 
essentially to confirm the logistics for the assessments. 



4.5.2 Conducting the Direct Child Assessments 

The direct child assessments were conducted from March through June 2004, the same time 
of year as in prior spring data collections. The school coordinator set the assessment date with a 
supervisor based on the school’s schedule. Over three-quarters of the assessments were completed by the 
end of April, with 20.5 percent completed in May and 1.3 percent completed in June. There was no 
evidence that certain types of children or schools were assessed early or late in the field period. 
Table 4-10 presents the weekly completion rates for the child assessments. In year-round schools, 
assessment teams made multiple visits to the school, visiting when each track was in session to assess the 
sampled children. There were 501 (2.3 percent) sampled children in year-round schools. 
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Table 4-10. Weekly completion of the child assessments: School year 2003-04 



Week # 


Week 

starting 


Completed 

assessments 1 


Cumulative 

completed 

assessments 


Cumulative 
Percent 
of total 


1 


2/25/04 


55 


55 


0.5 


2 


3/3/04 


666 


721 


6.3 


3 


3/10/04 


1,543 


2,264 


19.9 


4 


3/17/04 


1,080 


3,344 


29.4 


5 


3/24/04 


1,112 


4,456 


39.2 


6 


3/31/04 


1,074 


5,530 


48.6 


7 


4/7/04 


981 


6,511 


57.3 


8 


4/14/04 


666 


7,177 


63.1 


9 


4/21/04 


904 


8,081 


71.1 


10 


4/28/04 


804 


8,885 


78.2 


11 


5/6/04 


814 


9,699 


85.3 


12 


5/13/04 


704 


10,403 


91.5 


13 


5/20/04 


541 


10,944 


96.3 


14 


5/27/04 


279 


11,223 


98.7 


15 


6/2/04 


82 


11,305 


99.4 


16 


6/9/04 


33 


11,338 


99.7 


17 


6/16/04 


22 


11,360 


99.9 


18 


6/23/04 


8 


11,368 


100.0 



1 Based on field production reports. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



The direct child assessments were usually conducted in a school classroom or library. Before 
conducting the assessments, field supervisors and assessors set up the room for the assessments. They 
followed procedures for meeting children that were agreed upon during the preassessment contact with 
the school. Each child was signed out of his or her classroom prior to the assessment and signed back into 
the classroom upon the conclusion of the assessment. During the scheduling of schools in the fall, an 
attempt was made to conduct the direct child assessments at about the same point in time from the 
beginning of school year and the end of the year to increase the chances that exposure to instruction was 
about the same for all children. The fifth-grade direct child assessments averaged 97 minutes in length. 



Table 4-11 displays the number of completed child assessments for each round of data 
collection, including spring-fifth grade. All of the assessments in spring-fifth grade were completed in 
English. The majority of the assessments (83.0 percent) were completed in original schools, and 
interestingly, the number of assessments in transfer schools decreased in spring- fifth grade. Only 17 
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Table 4-11. Completed child assessments by round of data collection: School years 1998-99, 1999-2000, 2001-02, and 2003-04 





Fall- 

kindergarten 


Spring- 

kindergarten 


Fall- 

first grade 


Spring- 
first grade 




Spring- 
third grade 




Spring- 
fifth grade 






Number 


Percent 


Number Percent 


Number 


Percent 


Number Percent 


Number Percent 


Number Percent 


Child assessments completed 


19,147 


100 


19,987 


100 


5,297 


100 


16,622 


100 


14,502 


100 


11,368 


100 


Assessed 


In English, no accommodation 1 


17,019 


88.9 


18,342 


91.8 


4,848 


91.5 


15,460 


93.0 


13,565 


93.5 


10,813 


95.1 


In Spanish 


1,008 


5.3 


724 


3.6 


176 


3.3 


286 


1.7 


t 


t 


t 


t 


In other language 


410 


2.1 


229 


1.1 


33 


0.6 


37 


0.2 


t 


t 


t 


t 


With accommodation 1 


515 


2.7 


579 


2.9 


195 


3.7 


761 


4.6 


814 


5.6 


465 


4.1 


Excluded 


88 


0.5 


70 


0.4 


28 


0.5 


47 


0.3 


74 


0.5 


62 


0.5 


Partial complete 


107 


0.6 


43 


0.2 


17 


0.3 


31 


0.2 


49 


0.3 


28 


0.2 


Child assessments completed 


19,147 


100 


19,987 


100 


5,297 


100 


16,622 


100 


14,502 


100 


11,368 


100 


Original sampled school 


19,147 


100 


19,463 


97.4 


4,867 


91.9 


14,830 


89.2 


10,820 


74.6 


9,439 


83 


Transfer school 


0 


0 


524 


2.6 


430 


8.1 


1,792 


10.8 


3,682 


25.5 


1,929 


17 



'Not applicable. 

'The term accommodation in this table is the field operational definition of accommodation, which includes the wearing of glasses and hearing aids. These types of aids were systematically tracked to 
ensure that every child had the same chance at a successful assessment. With this information, assessors could prompt a child (e.g., to get her glasses before being assessed). 

NOTE: This table reflects final production numbers prior to statistical adjustment. This table does not include children who were subsampled out in fall- and spring-first grade and spring-third grade (see 
section 5.5.4.) These numbers should not be used to estimate student mobility. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), fall 1998, spring 1999, fall 1999, 
spring 2000, spring 2002, and spring 2004. 



percent of the sample was assessed in transfer schools in spring-fifth grade. Assessments of 
homeschooled children, or children in nonparticipating transfer schools, were held at the parent’s home or 
another location, such as a library, of the parent’s choosing. 



Less than 1 percent of participating children in fifth grade required accommodations or were 
excluded from the direct child assessments. Children were excluded from the direct assessments if they 
had a disability, e.g., blindness, or deafness, that could not be accommodated by the ECLS-K direct 
assessments, or their Individualized Education Program prevented their participation in assessments or 
required an accommodation not offered by the ECLS-K assessments. Accommodations offered by the 
ECLS-K assessments were as follows: alternative setting, scheduling, or timing; health care aide present; 
or the use of a personal assistive device. Table 4-12 presents the number of children excluded from or 
requiring an accommodation to the direct child assessment procedures in the spring of fifth grade. 



Table 4-12. Number of children excluded from or accommodated in the spring-fifth grade assessments: 
School year 2003-04 



Category 


Number of children 


Exclusions 


Excluded for disability 


63 


Accommodation 


Alternative setting accommodation 


50 


Scheduling/timing accommodation 


64 


Health care aide present 


12 


Personal assistive device 


9 



1 The term accommodation in this table includes only those accommodations offered during the assessment such as an alternative setting. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



4.5.3 Conducting the Parent Interview 

Parent interview procedures mirrored those of previous rounds of data collection. The parent 
interview was administered, primarily by telephone interview using CAI, between February and June 
2004. Slightly over 50 percent of the parent interviews were completed in February and March, 43 
percent were completed in April and May, and 6 percent were completed in June. The parent interview 
averaged 43 minutes. Table 4-13 presents the weekly completion of parent interviews. 
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Table 4-13. Weekly completion of parent interviews, spring-fifth grade: School year 2003-04 



Week 


Week 

starting 


Completed 

interviews 1 


Cumulative 

completed 

interviews 


Cumulative 
percent 
of total 


1 


2/25/04 


46 


46 


0.4 


2 


3/3/04 


675 


721 


6.6 


3 


3/10/04 


1,218 


1,939 


17.7 


4 


3/17/04 


1,286 


3,225 


29.5 


5 


3/24/04 


1,201 


4,426 


40.5 


6 


3/31/04 


1,083 


5,509 


50.4 


7 


4/7/04 


917 


6,426 


58.7 


8 


4/14/04 


737 


7,163 


65.5 


9 


4/21/04 


724 


7,887 


72.1 


10 


4/28/04 


660 


8,547 


78.1 


11 


5/6/04 


460 


9,007 


82.3 


12 


5/13/04 


456 


9,463 


8605 


13 


5/20/04 


464 


9,927 


90.7 


14 


5/27/04 


326 


10,253 


93.7 


15 


6/2/04 


215 


10,468 


95.7 


16 


6/9/04 


190 


10,658 


97.4 


17 


6/16/04 


147 


10,805 


98.8 


18 


6/23/04 


91 


10,896 


99.6 


19 


6/30/04 


44 


10,940 


100.0 



1 Based on field production reports from the parent interview. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



As in previous rounds of data collection, the parent interview was conducted in person if the 
respondent did not have a telephone. Table 4-14 contains the number of parent interviews per round, 
including spring-fifth grade. In fifth grade, only 2.7 percent of all completed parent interviews were 
conducted in person; 8.1 percent of all completed parent interviews were conducted in a language other 
than English; 95.1 percent of the latter were conducted in Spanish. 



In spring-fifth grade data, 10 percent of the parent cases were classified as final nonresponse. 
As in most field studies, the primary reasons for final nonresponse were parents who could not be located 
and parents who refused to complete the interview; 32.4 percent of the nonresponse parent cases were 
parents who could not be located, 43.9 percent were refusals and 23.8 percent were other nonresponse 
(e.g., language barrier). 
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Table 4-14. Number and percent of completed parent interviews by data collection mode, language, and wave of data collection: School years 
1998-99, 1999-2000, 2001-02, and 2003-04 





Fall- 

kindergarten 


Spring- 

kindergarten 


Fall- 

first grade 




Spring- 
first grade 




Spring- 
third grade 




Spring- 
fifth grade 






Number 


Percent 


Number 


Percent 


Number Percent 


Number Percent 


Number Percent 


Number Percent 


Parent interviews 


17,997 


100 


18,907 


100 


5,073 


100 


15,576 


100 


13,504 


100 


10,940 


100 


completed 


In person 


618 


3.4 


619 


3.3 


211 


4.2 


456 


2.9 


319 


2.4 


295 


2.7 


By phone 


17,379 


196.6 


18,288 


96.7 


4,862 


95.8 


15,120 


97.1 


13,185 


97.6 


10,645 


97.3 


Parent interviews 


17,997 


100 


18,907 


100 


5,073 


100 


15,576 


100 


13,504 


100 


10,940 


100 


completed 


English 


17,379 


96.6 


17,482 


92.5 


4,717 


93.0 


14,319 


91.9 


12,416 


91.9 


9,444 


90.9 


Spanish 


618 


3.4 


1,321 


7.0 


351 


6.9 


1,071 


6.9 


932 


6.9 


846 


7.7 


Other language 


0 


0 


81 


0.4 


0 


0 


75 


0.5 


41 


0.3 


39 


0.4 


Partial complete 


0 


0 


23 


0.1 


5 


0.1 


111 


0.7 


115 


09 


111 


1 



NOTE: This table completes final production numbers prior to statistical adjustment. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), fall 1998, spring 1999, 
fall 1999, spring 2000, spring 2002, and spring 2004. 



4.5.4 Conducting Data Collection on Children Who Withdrew From Their Previous Round 

School 

While contacting schools, field supervisors asked school coordinators to identify children 
who had withdrawn from the school since the spring of third grade. School staff were asked whether they 
knew the name and address of the school to which the child transferred, as well as any new information 
about the child’s household address. For the children who had moved from their spring-third grade school 
and were not part of the sample to be followed, information was collected only from the school personnel 
and not from parents. For children who had withdrawn from their spring-third grade school and were 
identified to be followed (i.e., were part of the sample of movers), supervisors also consulted parents and 
other contacts for information on the children’s new school. This information was entered into the FMS 
and processed at Westat for data collection. 

Table 4-15 presents the status of the children who were identified as movers in fifth grade; 
12,717 children were identified as having transferred from the school in which they were enrolled during 
the spring of base year, first grade, or third grade. The movers described in this table are defined as 
“operations movers” rather than “statistical movers” since cooperation must be secured from the transfer 
schools in order for data collection to proceed (see section 3.6.1 for a discussion of statistical movers.). 
There are two flags that describe movers. One flag is for statistical purposes and the other for operations 
purposes. The definitions for the two flags necessarily differ. For example, children who move into a 
destination school are flagged as a mover for operations purposes, since cooperation must be secured 
from the school to follow the child there, but the statistical flag is not set since these children are not 
movers in the statistical definition. Table 4-15 describes total movers across all rounds, which is the only 
way they were tracked; the chapter 3 tables describe the grade 5 movers from a statistical perspective. Of 
the 12,717 mover children in spring-fifth grade, 4,187 (32.9 percent) were in scope (i.e., children selected 
to be followed) and followed. The remaining 8,530 mover children were out of scope and were not 
followed; no child assessment or parent interview was conducted for these children. 
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Table 4-15. Number of children who moved in spring-fifth grade by completion category: School year 
2003-04 



Category 


Spring-fifth grade 




Number of children 


Percent 


Total movers 1 


12,717 


100.0 


Out-of-scope 2 


8,530 


67.1 


Did not follow 3 


7,880 


92.4 


Moved to outside of U.S. 3 


153 


1.8 


Deceased 3 


7 


<1 


Excluded from spring-fifth grade 4 


490 


5.7 


In-scope and followed 2 


4,187 


32.9 


Completed assessment 5 


3,299 


78.8 


Unlocatable 5 


281 


6.7 


Nonsampled primary sample unit 5 


395 


9.4 


Assessment refused 5 


149 


3.6 


Not assessed/absent 5 


63 


1.5 



1 The movers described in this table are defined as “operations movers" rather than “statistical movers” since cooperation must be secured from 
the transfer schools in order for data collection to proceed. 



2 Percent based on total movers. 

3 Percent based on out-of-scope children. 

4 In fifth grade, four groups of children were excluded, irrespective of other subsampling procedures that were implemented. They are (1) 
children who became ineligible in an earlier round (because they died or moved out of the country); (2) children who were subsampled out in 
previous rounds because they moved out of the original schools and were not subsampled to be followed; (3) children whose parents emphatically 
refused to cooperate (hard refusals) in any of the data collections rounds since spring-kindergarten; and (4) children in the third-grade sample for 
whom there are neither first-grade nor third-grade data. 

5 Percent based on in-scope children. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



Parent interviews were attempted for all in-scope children. However, different school and 
assessment data collection strategies were followed for children who moved, depending on where they 
moved and the cooperation status of their new school. School and assessment data collection was 
attempted for children who moved and were flagged as “follow” in spring-fifth grade in the following 
ways: 



■ Data collected for children moving into cooperating base year sampled schools 
included the child assessments in the school, school administrator questionnaire, 
regular and/or special education teacher questionnaires, facilities checklist, and 
student record abstract forms. 

■ Data collected for children moving into nonsampled schools in base year cooperating 
districts included the child assessments in the school, school administrator 
questionnaires, regular and/or special education teacher questionnaires, and student 
record abstract forms, if school permission was obtained. If school permission was not 
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obtained, the assessments were conducted in the home and no school or teacher data 
were collected. 

■ For children moving into transfer schools that refused, schools in sampled districts 
that refused, or originally sampled schools that were ineligible when sampled because 
they did not have kindergarten classes, the direct child assessments were conducted in 
the home. No school or teacher data were collected. 

■ For children moving into schools in nonsampled districts or dioceses: 

If the school was within the primary sampling unit (PSU), data collected 
included the child assessments in the school, school administrator 
questionnaire, regular and/or special education teacher questionnaires, facilities 
checklist, and student record abstract forms, if school permission was obtained. 
If school permission was not obtained, the assessments were conducted in the 
home and no school or teacher data were collected. 

If the school was outside a sampled PSU, no child, school, or teacher data were 
collected. 

■ For children who were not enrolled in school in the spring (including children who 
were home schooled), data collected included the child assessments in the home if the 
child was in the sampled PSU. If the child was outside the sampled PSU, no child 
assessment or school or teacher data were collected. 

As previously mentioned, 4,039 movers were identified during the fall preassessment 
contact. During spring-fifth grade, an additional 366 movers were identified. The identification of 92 
percent of the fifth-grade movers during the fall contact accomplished two important goals: (1) allowed 
additional time to trace movers; and (2) reduced the burden of tracing a large number of movers during 
the spring-fifth grade data collection. Table 4-16 presents the final status of the children who were 
identified as movers in fifth grade; a total of 12,717 children were identified as having transferred from 
the school in which they were enrolled when they were sampled in kindergarten. Of the 12,717 children 
identified as movers in spring-fifth grade, 4,187 children were selected to be followed and were followed 
(32.9 percent of total movers). The remaining 8,530 mover children were out-of-scope for this round of 
data collection because they moved out of the country, were deceased, excluded from fifth grade because 
they had no first- or third-grade data, or were subsampled out; no child assessments or parent interviews 
were conducted for these children. 
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Table 4-16. Number of children who moved by completion category: School years 1998-99, 1999-2000, 2001-02, and 2003-04 





Spring-kindergarten 


Spring- first grade 


Spring-third grade 


Spring-fifth grade 


Category 


Number 

of 

children 


Percent 


Number 

of 

children 


Percent 


Number 

of 

children 


Percent 


Number 

of 

children 


Percent 


Total movers 1 


1,568 


100 


5,759 


100 


9,889 


100 


12,717 


100.0 


Out of scope 2 


16 


1 


2,893 


50.2 


4,221 


42.7 


8,530 


67.1 


Did not follow 3 


0 


0 


2,848 


98.4 


4,102 


97.2 


7,880 


92.4 


Moved to outside of 
U.S. 3 


16 


1 


44 


1.5 


117 


2.7 


153 


1.8 


Deceased 3 


0 


0 


1 


<.l 


2 


<.l 


7 


<.l 


Excluded from fifth 
grade 


t 


t 


t 


t 


t 


t 


490 4 


5.7 


In-scope and followed 2 


1,552 


98.9 


2,866 


49.8 


5,668 


57.3 


4,187 


32.9 


Completed assessment 3 


510 


32.8 


1,792 


62.5 


3,682 


65.0 


3,299 


78.8 


Unlocatable 3 


501 


32.3 


271 


9.5 


607 


10.7 


281 


6.7 


Nonsampled primary 
sample unit 3 


282 


18.2 


454 


15.8 


871 


15.4 


395 


9.4 


Nonresponse 


259 


16.7 


349 


12.2 


508 


8.9 


212 


5.1 



f Not applicable. 

1 The movers described in this table are defined as “operations movers” rather than “statistical movers” since cooperation must be secured from the transfer schools in order for data collection to 
proceed. 

2 Percent based on total movers. 

3 Percent based on out-of-scope children. 

4 In fifth grade, four groups of children were excluded, irrespective of other subsampling procedures that were implemented. They are (1) children who became ineligible in an earlier round 
(because they died or moved out of the country); (2) children who were subsampled out in previous rounds because they moved out of the original schools and were not subsampled to be 
followed; (3) children whose parents emphatically refused to cooperate (hard refusals) in any of the data collections rounds since spring-kindergarten; and (4) children in the third-grade sample 
for whom there are neither first-grade nor third-grade data. 

5 Percent based on in-scope children. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 



Of the children who moved in fifth grade and were selected to be followed, 66.9 percent 
moved into nonsampled schools, 5.3 percent moved into a school outside the PSU and 10.4 percent of the 
movers could not be located. Table 4-17 presents the fifth-grade movers by school and district status. 



Table 4-17. Children moving in fifth grade: School year 2003-04 



Category 


Number 


Percent 


Total movers eligible and followed 


3,036 


100 


Moved into cooperating schools 


459 


15.1 


Moved into nonsampled schools 


2,030 


66.9 


Moved into schools/districts that refused 


27 


<1 


Home schooled 


41 


1.4 


Moved outside the primary sampling unit 


162 


5.3 


Moved — could not be located 


317 


10.4 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



4.5.5 Teacher and School Data Collection 

Data were collected from school administrators, regular classroom teachers, and special 
education teachers from February through June 2004. The school and teacher questionnaires were mailed 
to the school coordinators in January and February 2004. This schedule allowed additional time for these 
respondents to complete and return the instruments to Westat. 

During the fall advance school contact, it became apparent that many fifth-grade teachers 
would be asked to report about many sampled children and multiple domains (e.g., reading and math or 
science) due to the modification to the fifth-grade teacher questionnaires (see section 2.5 for a detailed 
discussion). In fifth grade, there was still considerable clustering of students in classrooms and teachers 
would be repeating classroom level information across many questionnaires. Table 4-18 presents the 
number of child-level questionnaires and percent of teachers asked to complete them based on fall 
linkages of children to teachers and domains as well as the percent of teachers completing that number of 
questionnaires during spring data collection. 
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Table 4-18. Number of child-level questionnaires and percent of teachers: School year 2003-04 



Number of child-level questionnaires to 
complete 


Percent of teachers 
expected from fall school 
contact 


Percent of teachers 
completing questionnaires 
from spring- fifth grade 


1 to 4 questionnaires 


63.0 


75.0 


5 to 1 0 questionnaires 


26.0 


18.0 


11 to 15 questionnaires 


6.0 


4.0 


16 to 46 questionnaires 


4.0 


2.0 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten 
Class of 1998-99 (ECLS-K), spring 2004. 



A procedure to reduce burden on teachers was developed to identify a “Key Child” for each 
domain and class. Teachers were asked to complete all items in the reading, mathematics, and science 
questionnaires for the designated key child, designated by a blue dot on the questionnaire cover, for the 
appropriate domains/classes. They were asked to complete only the child-level items, and not the 
classroom-level items, for the remaining children in that domain/class. 



Using the child-teacher-domain-class linkage information collected in the fall, customized 
packages of questionnaires were assembled for each child’s reading and mathematics or science teachers 
and/or special education teachers. The customized teacher questionnaire materials included the following: 

■ A Teacher Questionnaire cover letter; 

■ Teacher Questionnaire; 

■ A Reading Instruction Sheet attached to the child-level reading questionnaires for 
each separate reading class; 

■ A Math Instruction Sheet attached to the child-level math questionnaires for each 
separate math class; 

■ A Science Instruction Sheet attached to the child-level science questionnaires for each 
separate science class; and, 

■ A Special Education Instruction Sheet attached to the child-level special education 
questionnaires (if appropriate). 

The Teacher Questionnaire Cover letter was a much simplified document from previous 
rounds. It laid out the teacher tasks in succinct language on one page and clearly described the 
questionnaires, including the questionnaire cover colors, to help the teacher easily identify each part of 
the package. A checklist of instructions, an easy step-by-step guide, was included to aid the teacher in 
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completing the questionnaires and returning them to the school coordinator. Domain instruction sheets 
(Reading, Mathematics, Science, and Special Education) included all the information the teacher required 
to correctly identify the class/domain. Teachers only received instruction sheets required for the sampled 
children in the domains they taught. All questionnaires were labeled with names and ID numbers. Teacher 
questionnaires were labeled with teacher name and ID number. Child questionnaires were labeled with: 
teacher name and ID number, child name and ID number, domain, reading, mathematics, or science; 
classroom; and class time. The key domain child questionnaires were labeled with a blue dot. Child-level 
questionnaires were assembled in bundles by domain and class identifier (e.g., reading, 9:15 am, 
mathematics, Room A) for each teacher. All materials and questionnaires for each teacher were placed in 
an envelope and the envelope labeled with the teacher name. 

Teacher and school administrator packets were bundled together by school and mailed to the 
school coordinator for distribution. If the school and/or teacher and school administrator were not 
identified in the fall advance contact, then the supervisor gathered the relevant information during the 
preassessment call in the spring and mailed the packets at that time. During their visits to the schools, 
field supervisors also completed a facilities checklist for each sampled school. 

Field supervisors began prompting for the return of questionnaires when they contacted 
schools to confirm the assessment schedule. During the field period, field supervisors followed up with 
school administrators and teachers by telephone and with visits to the schools to prompt for the return of 
the questionnaires. Field supervisors returned completed questionnaires to Westat. 

In April 2004, the field supervisors were instructed to conduct intensive followup for 
missing school administrator questionnaires for schools with high minority student populations to reduce 
potential bias. There were 334 such schools identified. As a result of the followup efforts by the field 
supervisors, 324 of the 334 schools (97 percent) completed and returned the school administrator 
questionnaires. 



4.5.5. 1 Hard-Copy Data Retrieval 

Retrieval Procedure. Data retrieval involved collecting missing items for some 
questionnaires that were otherwise complete. Based on analyses of the success of the third-grade data 
retrieval, critical items were determined only for the school administrator questionnaire and the Key 
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Domain child questionnaires for each class. The approach to retrieval for the school administrator 
questionnaire and child-level teacher questionnaires called for scanning and retrieval in the school. School 
coordinators, administrators, and teachers were informed in their letters that the supervisors would collect 
completed questionnaires on assessment day. No mailer for returning the questionnaires to the home 
office was provided to the schools. Respondents were asked to complete the instruments, seal them in 
envelopes, and give them to the school coordinator. Supervisors scanned the school administrator 
questionnaire and Key Child domain questionnaires on-site, and conducted retrieval as necessary. Exhibit 
4-19 presents the critical items by questionnaire. Each of the teacher questionnaire items proposed for 
retrieval were completed in the Key Child instruments only, since these are class-level items. 

Exhibit 4-19. Critical items by questionnaire type: School year 2003-04 



Instrument 


Item 


Question Number 


School administrator (SAQ) 


Grade level 


Question 4 


School type/control 


Questions 5, 6, 7 




Race/ethnicity 
School lunch program 


Question 8 




eligibility/participation 


Question 14 


Key Domain child-level reading 


Grades taught 


Question 15 


teacher questionnaires 


Race/ethnicity 


Question 16 




Sex 


Question 17 




Gifted/talented, LEP, Disability, Absent 


Question 18 


Key Domain child -level 


Grades taught 


Question 5 


mathematics teacher 


Race/ethnicity 


Question 6 


questionnaires 


Sex 


Question 7 




Gifted/talented, LEP, Disability, Absent 


Question 8 


Key Domain child -level science 


Grades taught 


Question 4 


teacher questionnaires 


Race/ethnicity 


Question 5 




Sex 


Question 6 




Gifted/talented, LEP, Disability, Absent 


Question 7 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



Data Retrieval Attempts. Field supervisors attempted to retrieve missing critical items and 
missing questionnaires in the schools in their assignments. They scheduled their retrieval efforts for the 
day the assessments were scheduled and attempted to find the respondents in person. Otherwise, they 
attempted these retrieval attempts by telephone. Field supervisors recorded any changes to missing critical 
items in blue pencil in the questionnaire. If the respondent did not know the answer, they recorded “DK” 
by the item; if the respondent refused to answer, they recorded “RF” by the item. Field supervisors 
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recorded the status of the questionnaire as one of the following: 



■ Questionnaire complete with no missing critical items: no data retrieval required. 

■ Questionnaire is missing one or more critical items: data retrieval required; one or 
more critical items collected. 

■ Questionnaire is missing one or more critical items: data retrieval required; no critical 
items were collected. 

■ Questionnaire refused: unit nonresponse. 



Table 4-19 presents the results of attempts to retrieve critical items. 



4.5.6 Incentives in the ECLS-K 

In order to gain respondent cooperation and ensure participation throughout the various data 
collection phases of the ECLS-K study, various incentives were offered. The type of incentive, monetary 
or nonmonetary, depended on whether the respondent was a sampled child, parent, teacher, or school. 
Exhibit 4-20 delineates the types of incentives used in the ECLS-K. 

Child Incentives. Children were given a small token at the end of the assessment to thank 
them for their cooperation in completing the assessment. In the spring- fifth grade, they were given a green 
lanyard with the ECLS-K sun log and study name. In addition, each month Westat mailed birthday cards 
to children whose birthdays fell within that month. Children were sent birthday cards throughout the 
calendar year, not just during the school year. By mailing these cards, children were not only thanked 
again for their participation, but parents were also reminded about the study. These periodic reminders are 
important in a longitudinal study, in which respondents may become apathetic toward the study during 
later rounds. Not only do these reminders encourage respondent participation, but they help the home 
office update addresses of families that have moved. 
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Table 4-19. Total number and percent of the final results of critical item retrieval: School year 2003-04 





Cases needing no data retrieval 


Data retrieval 
complete 


Data retrieval refused 


Refused 




Questionnaire type 


Total 

expected 1 


Number 


Percent 2 


Number 


Percent 2 


Number 


Percent 3 


Number Percent 2 


School administrator 


2,442 


607 


24.9 


472 


19.3 


1,045 


42.8 


318 


13.0 


Reading teacher 
questionnaire 


12,308 


9,737 


79.1 


625 


5.1 


517 


4.2 


683 


5.5 


Mathematics teacher 
questionnaire 


5,751 


4,743 


82.5 


322 


5.6 


317 


5.5 


369 


6.4 


Science teacher 
questionnaire 


5,811 


4,867 


83.8 


273 


4.7 


305 


5.2 


366 


6.3 



1 This column represents the number of questionnaires expected from school and teacher respondents based on schools and teachers linked to sampled children. 

2 Percent based on number expected. 

J Percent based on number complete, needing data retrieval. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 



Exhibit 4-20. Types of incentives used in the ECLS-K: School year 2003-04 



Respondent 


Incentive 


Child 


■ 

■ 


ECLS-K lanyard 
Birthday card 


Parent 


■ 

■ 


Newsletters 

ECLS-K magnet with notepad 


Teacher 


■ 

■ 


$20 mailed with questionnaire package 

$7 for each completed child- level questionnaire 


School — original 


■ 

■ 

■ 


$50 mailed with school administrator questionnaire package 

$150 for cooperating schools 

$7 for each completed student records abstract 


School — transfer 


■ 

■ 


$50 mailed with school administrator questionnaire package 
$7 for each completed student records abstract 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



Parent Incentives. In the fall of 2003, a newsletter about the study was published and 
mailed to parents. The newsletter served to update respondents on the initial findings from the first grade 
year and inform them about the future rounds of data collection. Parent involvement with homework, 
teacher views on homework and the average time spent on homework were a few of the topics discussed. 
Not only did the newsletter update parents on the findings of the study and highlight its importance, but it 
also was an incentive for future rounds of participation. Respondents were able to see the results of their 
participation in the study. 



Parents received an incentive for participating in the parent interview. At the end of the field 
period, thank-you letters were generated for every respondent who completed a parent interview. The 
letters, along with an ECLS-K magnet with attached notepad, were mailed to the respondents. Letters 
translated into Spanish were mailed to those respondents who completed the interview in Spanish. 

Teacher Incentives. In order to maximize response rates, all teachers were mailed a $20 
check with the questionnaire package. Teachers were also reimbursed $7 for each child for whom they 
completed a questionnaire, regardless of the number of questionnaires. Special education teachers were 
reimbursed in the same manner. 



School Incentives. Schools were also paid a monetary incentive for participating in the 
ECLS-K. Because school staff are often very busy and may not be aware of the benefits of cooperating, 
the cooperating original sample schools were remunerated $200 for participating, of this $50 was mailed 
with the school administrator questionnaire package and $150 was mailed once the school assessment 
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work was begun. Transfer schools received the $50 with the school administrator questionnaire package. 
School staff who completed the student records abstract received $7 for each questionnaire completed. 



All checks to schools, teachers, and school staff completing the SRAs were mailed weekly 
during the field period and were sent with thank-you letters. The checks to teachers and to staff 
completing the SRA were made out to the individual who completed the forms. 



4.6 Data Collection Quality Control 

The ECLS-K data are used by researchers to study children’s school experience and its 
relation to student outcomes, and by educators and policymakers to inform policy decisions. It is 
important that the information used by these groups is based on sound research practice and that 
considerable attention be paid to identifying potential sources of error, quantifying this error, and 
designing techniques to either reduce the error or minimize its impact on survey estimates. The work 
carried out in support of the ECLS-K includes a variety of activities that are directed toward ensuring that 
the data are of high quality. 



4.6.1 Maintaining Reliability on the Child Assessment 

To ensure that assessors maintained the standard that they achieved at training, assessors 
were observed by their supervisor in the field at two different points in time. The first observation was to 
be conducted by the end of March and the second observation by the end of April. The supervisor 
completed the Assessment Observation Form, which rated the assessor on key areas of the assessment 
protocol. In the Assessment Observation Form, the supervisor simultaneously coded with the assessor 
those open-ended assessment items that required judgment by the assessor to determine whether the 
child’s answer was correct. At the end of the assessment period, after the child was escorted from the 
room, the supervisor and the assessor reviewed the assessor’s overall performance. The two also 
compared the way that they each handled the open-ended questions. If there were large discrepancies, 
they reviewed the QxQs for these items carefully. 
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4.6.1. 1 



Assessment Observation Form 



The Assessment Observation Form had the names of the assessor and the supervisor, the 
case ID observed, the observation number, as well as the date the observation was conducted. The form 
had two sections: section 1 (shown in exhibit 4-21) was used by supervisors to rate the assessor on key 
overall skill areas, such as building rapport, using neutral praise, responding to behaviors presented by the 
child, pacing appropriately, and coaching. In section 1 the supervisor checked “No” for each skill area 
that the assessor did not demonstrate appropriately. 



Exhibit 4-21. Section 1 of the Assessment Certification Form: School year 2003-04 



Evaluator: As the assessment is administered, record whether or not the assessor successfully 
performed the following behaviors. Check “No” if the assessor makes 3 or 4 errors and needs to 
make improvements. 



SECTION 1 : Rapport building and working with the child 



Q: 


Did the assessor... 


No 


1 . 


Establish rapport with the child? 




2. 


Use appropriate response to DK responses from the child? 




3. 


Use neutral praise? 




4. 


Respond to behaviors presented by child? 




5. 


Avoid coaching the child? 




6. 







SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



Section 2 listed specific questions from each routing and subdomain (e.g., mathematics) 
form. The instructions for completing section 2 are shown in exhibit 4-22. For each of the listed questions 
observed, the supervisor recorded both the child’s response and if the assessor did not demonstrate the 
specified required administration skills for that question. The required administration skills included 
reading questions verbatim, using appropriate probes, and using appropriate hand motions. For each 
question in which the supervisor observed that the assessor did not demonstrate the required 
administration skill(s), he or she checked a box, indicating which skill was not performed. 
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Exhibit 4-22. Instructions for Section 2 of the Assessment Observation Form: School year 2003-02 



SECTION 2: Specific Assessment Activities 

Supervisor/Evaluator: Code the items as the assessor administers the assessment. 

Code the child’s response as the item is administered. 

If the item requires probing, check the box if the assessor does not use the appropriate probe. 

Check the box in the “Verbatim” column if the assessor does not read the item exactly as worded on the 
screen. 

Check the box in the “Gesturing” column if the assessor does not use appropriate hand motions. 

For each validation code item, check the box in the “Validation” column if the response coded by the 
assessor is not what you have coded. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



As described earlier, during the observed assessment, the supervisor coded the child’s 
response to those open-ended assessment items that required judgment on the part of the assessor. These 
open-ended items were flagged in the CAI program for quality control review, and a screen showing how 
the assessor coded each of the questions was accessed at the end of the assessment for review with the 
supervisor. The supervisor and assessor compared their codes to these open-ended questions. For each 
item where the coding did not match, the supervisor recorded this discrepancy by checking the 
corresponding box for that question in the Assessment Observation Form. 

Exhibit 4-23 shows an example of a question from the Assessment Observation Form, which 
required the supervisor to evaluate the required assessment administration skills: reading verbatim, using 
appropriate gesturing, using appropriate probe, and accurately coding the child’s response to this open- 
ended question. The supervisor was instructed to check the boxes only if the assessor did not demonstrate 
the correct behavior. For all questions that are observed by the supervisor and listed in the Assessment 
Observation Form, the supervisor recorded if the assessor did not read verbatim or gesture appropriately. 
Supervisors only recorded if assessors did not use the appropriate probe for those questions that display 
probes (a box labeled “Assessor Used Inappropriate Probe?” was presented in the Assessment 
Observation Form). Also, the supervisors only recorded if their response coding did not match the 
assessors’ for those questions that are open-ended (a box labeled “Validation Code Did Not Match” was 
presented in the Assessment Observation Form). 
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Exhibit 4-23. Sample question from Assessment Certification Form: School year 2003-04 





DID NOT READ 
QUESTION 
VERBATIM 


GESTURED 

INAPPRO- 

PRIATELY 


VALIDATION 
CODE DID 
NOT MATCH 


R5.245 What do you think is the difference between the 
storyteller’s bedroom and the sister’s bedroom?** 

IF STUDENT SAYS, “ A mess” OR “A lot of junk”, 
PROMPT: “How is it different?” 

CORRECT 1 

STORY TELLER’S BEDROOM IS: 
(BIGGER MESS/MORE 
MESS/MESSIER) 

(MORE JUNK/JUNKIER) 

(IS A MORE DISGUSTING 
SPOT) 

CORRECT ANSWER MUST 
INVOLVE A COMPARISON 

INCORRECT 2 

(STORY TELLER’S BEDROOM IS 

CLEAN/AS MESSY AS SISTER’S BEDF 
DETAILS ABOUT THE SISTER’S 
BEDROOM 
EXAMPLES: 

(PIZZA UNDER THE DESK) 

(SOCKS ON THE LIGHT) 
(MESS/ALSO A MESS) 

(JUNK/ A LOT OF JUNK) 

REFUSED 777 

DON’T KNOW 999 

ASSESSOR USED INAPPROPRIATE PROBE? 


□ 


□ 


□ 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



At the end of the child assessment, after the child was escorted from the room, the supervisor 
and the assessor reviewed the assessor’s overall performance. After discussing the ratings on section 1, 
the assessor accessed the quality control screen. The assessor and supervisor reviewed their codes for 
each open-ended question asked in section 2 of the Assessment Observation Form. 



The supervisor then scored the assessment observation using the scoring form shown in 
exhibit 4-24. In the first part of the form (Form A), the supervisor counted the number of check marks 
recorded in section 1 : Rapport Building and recorded that number in the appropriate row. The supervisor 
then recorded the number of check marks for each section of the assessment that was observed and 
entered those numbers in the appropriate box. 
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Exhibit 4-24. Observation Scoring Form: School year 2003-04 



Observation Scoring 

Step 1: Record the number of check marks from Section 1: Rapport Building and Section 2: Specific 

Assessment Activities in the appropriate boxes of Form A. 

Step 2: Sum each row and record total in Row Totals column of Form A. 

Step 3: Sum Row Totals column (including number of check marks from Section 1) and record in Total 

check marks box of Form A. 



Form A: Checks Marks Recorded in Sections 1 and 2 



Assessment 


Routing Form 


Red Form 


Yellow Form 


Blue Form 


Row Totals 


Reading 












Mathematics 












Science 












Number of check marks from Section 1: Rapport Building: 




Total Check Marks: 





Step 4: Using Form B, circle the number in the form used for each assessment. For example, if the Red 

Form was used for reading, circle the number 6 in the Routing Row and the number 20 in the 
Reading row under Red Form. Record each circled number in the Row Totals column of Form 
B. 

Step 5: Sum Row Totals column (including Possible Points for Section 1 ) and record in Total Possible 

Points box of Form B. 



Form B: Total Possible Points 



Assessment 


Routing Form 


Red Form 


Yellow Form 


Blue Form 


Row Totals 


Reading 


6 


20 


21 


20 




Mathematics 


8 


11 


18 


23 




Science 


11 


14 


15 


18 




Possible Points for Section 1: Rapport Building: 


6 


Total Possible Points: 





Step 6: Use Proportion Correct Chart to determine the proportion correct and write that proportion in 

this box: 

Result: If Proportion correct is greater than or equal to .85, Assessor is Certified. 

If Proportion correct is between .70 and .85, Assessor requires Remedial Action (Complete 
Remedial Task Chart below) 

If Proportion correct is less than .70, Assessor Failed. 



See note at end of exhibit. 
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Exhibit 4-24. Observation Scoring Form: School year 2003-04 — Continued 



Remedial Task Chart 

Circle the number of the task(s) needing improvement. Discuss remedial action with the assessor. 



TASK 


# 


REMEDIAL ACTION 


APPROPRIATE PACE 


1 


PRACTICE/REVIEW JOB AIDS 


AVOID COACHING 


2 


REVIEW THE JOB AIDS 


READ VERBATIM 


3 


PRACTICE READING FROM THE SCREEN 


GESTURING APPROPRIATELY 


4 


PRACTICE GESTURING 


USE APPROPRIATE PROBES 


5 


REVIEW THE JOB AIDS/Q-BY-Qs 


VALIDATION CODE DID NOT MATCH 


6 


REVIEW THE Q-BY-Qs 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



Exhibit 4-25 shows an example of a completed Form A. In this example, the supervisor 
recorded one (1) check mark for section 1, zero (0) check marks for Reading Routing, zero (0) check 
marks for Reading Blue, one (1) check mark for Math Routing, zero (0) check marks for Math Yellow, 
zero (0) check marks for Science Routing, and one (1) check mark for Science Red. The total check 
marks recorded was three (3). 

Exhibit 4-25. Example of Form A: School year 2003-04 



Form A: Checks Marks Recorded in Sections 1 and 2 



Assessment 


Routing Form 


Red Form 


Yellow Form 


Blue Form 


Row Totals 


Reading 


0 






0 


0 


Mathematics 


1 




0 




1 


Science 


0 


1 






1 


Number of check marks from section 1: Rapport Building: 


1 


Total Check Marks: 


3 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, 
Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 



The supervisor then calculated the number of possible points for the sections of the 
assessment observed using the second section of the scoring form (Form B) (exhibit 4-26). Continuing 
with the example started earlier, the supervisor observed all of Reading Blue (20 possible points), Math 
Yellow (18 possible points), and Science Red (14 possible points). In addition, section 1 and all the 
Routing Form s contribute 3 1 possible points. Based on the sections of the assessment that the supervisor 
observed the total number of possible points for this assessor was 83. 
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Exhibit 4-26. Example of Form B: School year 2003-04 



Form B: Total Possible Points 



Assessment 


Routing Form 


Red Form 


Yellow Form 


Blue Form 


Row Totals 


Reading 


6 


20 


21 


20 




Mathematics 


8 


11 


18 


23 




Science 


11 


14 


15 


18 




Possible Points for Section 1: Rapport Building: 


6 


Total Possible Points: 





SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten 
Class of 1998-99 (ECLS-K), spring 2004. 



Finally, the supervisor determined the proportion correct by using the Proportion Correct 
Chart displayed in exhibit 4-27. Continuing the example, the supervisor found the column on the 
Proportion Correct Chart displaying the total possible points for the assessor based on the portions of the 
assessment observed (74) and the row on the Proportion Correct Chart displaying the total check marks 
the supervisor recorded for the trainee (3), and recorded the proportion from the corresponding box on the 
Proportion Correct Chart in the box under step 6 (exhibit 4-28) on the scoring form (.96). 

Once the supervisor completed the scoring, the assessor was rated as Passed, Remedial 
Action, or Failed. 

The field supervisors recorded their observations on the form and then reviewed the form 
with the assessor. The most frequent problems observed were not reading the items verbatim and 
inappropriate gesturing. Feedback was provided to the assessors on the strengths and weaknesses of their 
performance and, when necessary, remedial training was provided in areas of weakness. Table 4-20 
presents the result of the observations. As table 4-20 shows, all assessors received scores of 85 percent or 
better while being observed. 
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Exhibit 4-27. Proportion Correct Chart: School year 2003-04 





69 


70 


72 


73 


74 


76 


77 


78 


79 


80 


81 


82 


83 


84 


85 


86 


87 


88 


89 


90 


92 


93 


1 


0.99 


0.99 


0.99 


0.99 


0.99 


0.99 


0.99 


0.99 


0.99 


0.99 


0.99 


0.99 


0.99 


0.99 


0.99 


0.99 


0.99 


0.99 


0.99 


0.99 


0.99 


0.99 


2 


0.97 


0.97 


0.97 


0.97 


0.97 


0.97 


0.97 


0.97 


0.97 


0.98 


0.98 


0.98 


0.98 


0.98 


0.98 


0.98 


0.98 


0.98 


0.98 


0.98 


0.98 


0.98 


3 


0.96 


0.96 


0.96 


0.96 


0.96 


0.96 


0.96 


0.96 


0.96 


0.96 


0.96 


0.96 


0.96 


0.96 


0.96 


0.97 


0.97 


0.97 


0.97 


0.97 


0.97 


0.97 


4 


0.94 


0.94 


0.94 


0.95 


0.95 


0.95 


0.95 


0.95 


0.95 


0.95 


0.95 


0.95 


0.95 


0.95 


0.95 


0.95 


0.95 


0.95 


0.96 


0.96 


0.96 


0.96 


5 


0.93 


0.93 


0.93 


0.93 


0.93 


0.93 


0.94 


0.94 


0.94 


0.94 


0.94 


0.94 


0.94 


0.94 


0.94 


0.94 


0.94 


0.94 


0.94 


0.94 


0.95 


0.95 


6 


0.91 


0.91 


0.92 


0.92 


0.92 


0.92 


0.92 


0.92 


0.92 


0.93 


0.93 


0.93 


0.93 


0.93 


0.93 


0.93 


0.93 


0.93 


0.93 


0.93 


0.93 


0.94 


7 


0.90 


0.90 


0.90 


0.90 


0.91 


0.91 


0.91 


0.91 


0.91 


0.91 


0.91 


0.91 


0.92 


0.92 


0.92 


0.92 


0.92 


0.92 


0.92 


0.92 


0.92 


0.92 


8 


0.88 


0.89 


0.89 


0.89 


0.89 


0.89 


0.90 


0.90 


0.90 


0.90 


0.90 


0.90 


0.90 


0.90 


0.91 


0.91 


0.91 


0.91 


0.91 


0.91 


0.91 


0.91 


9 


0.87 


0.87 


0.88 


0.88 


0.88 


0.88 


0.88 


0.88 


0.89 


0.89 


0.89 


0.89 


0.89 


0.89 


0.89 


0.90 


0.90 


0.90 


0.90 


0.90 


0.90 


0.90 


10 


0.86 


0.86 


0.86 


0.86 


0.86 


0.87 


0.87 


0.87 


0.87 


0.88 


0.88 


0.88 


0.88 


0.88 


0.88 


0.88 


0.89 


0.89 


0.89 


0.89 


0.89 


0.89 


11 


0.84 


0.84 


0.85 


0.85 


0.85 


0.86 


0.86 


0.86 


0.86 


0.86 


0.86 


0.87 


0.87 


0.87 


0.87 


0.87 


0.87 


0.88 


0.88 


0.88 


0.88 


0.88 


12 


0.83 


0.83 


0.83 


0.84 


0.84 


0.84 


0.84 


0.85 


0.85 


0.85 


0.85 


0.85 


0.86 


0.86 


0.86 


0.86 


0.86 


0.86 


0.87 


0.87 


0.87 


0.87 


13 


0.81 


0.81 


0.82 


0.82 


0.82 


0.83 


0.83 


0.83 


0.84 


0.84 


0.84 


0.84 


0.84 


0.85 


0.85 


0.85 


0.85 


0.85 


0.85 


0.86 


0.86 


0.86 


14 


0.80 


0.80 


0.81 


0.81 


0.81 


0.82 


0.82 


0.82 


0.82 


0.83 


0.83 


0.83 


0.83 


0.83 


0.84 


0.84 


0.84 


0.84 


0.84 


0.84 


0.85 


0.85 


15 


0.78 


0.79 


0.79 


0.79 


0.80 


0.80 


0.81 


0.81 


0.81 


0.81 


0.81 


0.82 


0.82 


0.82 


0.82 


0.83 


0.83 


0.83 


0.83 


0.83 


0.84 


0.84 


16 


0.77 


0.77 


0.78 


0.78 


0.78 


0.79 


0.79 


0.79 


0.80 


0.80 


0.80 


0.80 


0.81 


0.81 


0.81 


0.81 


0.82 


0.82 


0.82 


0.82 


0.83 


0.83 


17 


0.75 


0.76 


0.76 


0.77 


0.77 


0.78 


0.78 


0.78 


0.78 


0.79 


0.79 


0.79 


0.80 


0.80 


0.80 


0.80 


0.80 


0.81 


0.81 


0.81 


0.82 


0.82 


18 


0.74 


0.74 


0.75 


0.75 


0.76 


0.76 


0.77 


0.77 


0.77 


0.78 


0.78 


0.78 


0.78 


0.79 


0.79 


0.79 


0.79 


0.80 


0.80 


0.80 


0.80 


0.81 


19 


0.72 


0.73 


0.74 


0.74 


0.74 


0.75 


0.75 


0.76 


0.76 


0.76 


0.77 


0.77 


0.77 


0.77 


0.78 


0.78 


0.78 


0.78 


0.79 


0.79 


0.79 


0.80 


20 


0.71 


0.71 


0.72 


0.73 


0.73 


0.74 


0.74 


0.74 


0.75 


0.75 


0.75 


0.76 


0.76 


0.76 


0.76 


0.77 


0.77 


0.77 


0.78 


0.78 


0.78 


0.78 


21 


0.70 


0.70 


0.71 


0.71 


0.72 


0.72 


0.73 


0.73 


0.73 


0.74 


0.74 


0.74 


0.75 


0.75 


0.75 


0.76 


0.76 


0.76 


0.76 


0.77 


0.77 


0.77 


22 


0.68 


0.69 


0.69 


0.70 


0.70 


0.71 


0.71 


0.72 


0.72 


0.73 


0.73 


0.73 


0.73 


0.74 


0.74 


0.74 


0.75 


0.75 


0.75 


0.76 


0.76 


0.76 


23 


0.67 


0.67 


0.68 


0.68 


0.69 


0.70 


0.70 


0.71 


0.71 


0.71 


0.72 


0.72 


0.72 


0.73 


0.73 


0.73 


0.74 


0.74 


0.74 


0.74 


0.75 


0.75 


24 


0.65 


0.66 


0.67 


0.67 


0.68 


0.68 


0.69 


0.69 


0.70 


0.70 


0.70 


0.71 


0.71 


0.71 


0.72 


0.72 


0.72 


0.73 


0.73 


0.73 


0.74 


0.74 


25 


0.64 


0.64 


0.65 


0.66 


0.66 


0.67 


0.68 


0.68 


0.68 


0.69 


0.69 


0.70 


0.70 


0.70 


0.71 


0.71 


0.71 


0.72 


0.72 


0.72 


0.73 


0.73 


26 


0.62 


0.63 


0.64 


0.64 


0.65 


0.66 


0.66 


0.67 


0.67 


0.68 


0.68 


0.68 


0.69 


0.69 


0.69 


0.70 


0.70 


0.70 


0.71 


0.71 


0.72 


0.72 


27 


0.61 


0.61 


0.63 


0.63 


0.64 


0.64 


0.65 


0.65 


0.66 


0.66 


0.67 


0.67 


0.67 


0.68 


0.68 


0.69 


0.69 


0.69 


0.70 


0.70 


0.71 


0.71 


28 


0.59 


0.60 


0.61 


0.62 


0.62 


0.63 


0.64 


0.64 


0.65 


0.65 


0.65 


0.66 


0.66 


0.67 


0.67 


0.67 


0.68 


0.68 


0.69 


0.69 


0.70 


0.70 


29 


0.58 


0.59 


0.60 


0.60 


0.61 


0.62 


0.62 


0.63 


0.63 


0.64 


0.64 


0.65 


0.65 


0.65 


0.66 


0.66 


0.67 


0.67 


0.67 


0.68 


0.68 


0.69 


30 


0.57 


0.57 


0.58 


0.59 


0.59 


0.61 


0.61 


0.62 


0.62 


0.63 


0.63 


0.63 


0.64 


0.64 


0.65 


0.65 


0.66 


0.66 


0.66 


0.67 


0.67 


0.68 



NOTE: Proportion Correct = (Total Possible Points - Number of Check Marks)/Total Possible Points 

EXAMPLE: If an Assessor received 8 check marks out of a total of 86 possible points, the proportion correct is 0.91. This Assessor passed. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 




Exhibit 4-28. Example of Proportion Correct Chart: School year 2003-04 



Step 6: Use Proportion Correct Chart to determine the proportion correct and write that proportion in 


this box: .96 


.98 





SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



Table 4-20. Results of the child assessment observations, fifth-grade data collection: School year 
2003-04 



Number of observations 1 



Score on certification form 



Total: 466 
465 
1 
0 



85 percent or above 
70-84 percent 
Below 70 percent 



Two hundred and forty-two assessors were to be observed after training; assessors completing only parent interviews were not observed. 12 
assessors had only one observation; 227 assessors had two observations. Only 1 assessor failed to pass the first observation and was released from 
the project by mutual consent. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



4.6.1.2 Assessor Interrater Reliability 

As part of the child assessment observations described in section 4.6.1, field supervisors 
completed an assessment certification form for each observation they conducted. An important element of 
this form was the “validation items.” With the exception of the reading routing test, all of the assessments 
included at least one item that both the observer and the assessor scored. These items had open-ended 
responses that called for interpretation on the part of the assessor to determine whether a child’s response 
was correct. By comparing the extent to which assessors and observers agreed on scoring these validation 
items, a measure of interrater reliability was obtained. Interrater reliability provided a measure of the 
accuracy of the assessor’s scoring compared with the standard, the observer’s. 

Table 4-21 contains the results of these comparisons. As can be seen, overall interrater 
reliability was very high throughout all the forms. It was highest for mathematics (98 percent or better 
depending upon the form) and lowest for reading, with the reading yellow level (the medium reading 
level) showing the lowest percent agreement (95.7 percent). The reading yellow level path received a 
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relatively large number of observations (232) and also contained a relatively large number of validation 
items (5) compared with some of the other paths. Thus, there was greater opportunity for disagreement on 
this path compared with the others. The science blue level (the high science level) also had a relatively 
higher opportunity for disagreement (212 observations and 4 validation codes) and it, too, exhibited a 
somewhat lower interrater reliability (96.7 percent) compared with some of the other paths. The 
reliability, however, even on these more difficult paths, was high and demonstrated that the assessors 
accurately coded open-ended items. 



Table 4-21. Interrater reliability on child assessment validation items: School year 2003-04 



Category 


Number of 
Observations 


Number of 
Validation items 


Percent agreement: 
Assessors and observers 1 


Reading 


458 


13 


96.5 


Routing 


t 


0 


t 


Red 


103 


3 


97.1 


Yellow 


232 


5 


95.7 


Blue 


123 


5 


97.4 


Mathematics 


461 


10 


99 


Routing 


461 


2 


99.7 


Red 


162 


1 


100 


Yellow 


161 


3 


98.8 


Blue 


138 


4 


98.4 


Science 


463 


14 


96.8 


Routing 


460 


3 


99.1 


Red 


53 


4 


98.1 


Yellow 


198 


3 


96.6 


Blue 


212 


4 


96.7 



+ Not applicable. 



1 Percent agreement was calculated as follows: number of validation items observed in which observer agreed with the assessor divided by 
number of validation items observed. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 
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4.6.2 



Validation of Parent Interviews 



Approximately 10 percent of the respondents who completed parent interviews were called 
back by a field supervisor (i.e., validated). The first parent interview completed by an assessor was always 
validated. Over the course of the field period, a running count of an assessor’s completed parent 
interviews was maintained, and each tenth completed parent interview was selected for validation, thus 
ensuring that 10 percent of each assessor’s cases were selected for validation. The parent validation was 
approximately 5 minutes long and was conducted by telephone. No in-person cases were validated. 

Field supervisors used a standardized parent validation script, the Parent Interview 
Validation Form, to make validation calls to parents. The validation script included verification of the 
child’s name, date of birth, and sex, and 7 questions from the parent interview. Parent Validation 
Interview Forms were generated throughout the field period. By the end of June, nearly 10 percent 
(1,023) of the total completed parent interviews had been validated. Validation results were as follows: 

(1) “No changes” meaning responses to the original interview and the validation interview were identical; 

(2) “Minor changes” meaning there was a minor discrepancy (e.g., the ZIP code was different) between 
the responses to the original interview and the validation interview; and (3) “Major changes” meaning 
there was a discrepancy between the response to the original interview and the response in the validation 
interview (e.g., bed times varied between the two contacts). Table 4-22 presents the results of the parent 
validations. 



4.6.3 Validations of School Visits 

To ensure that assessments proceeded smoothly, a validation call was completed with the 
school principal in at least two of each supervisor’s assigned original schools in the spring-fifth grade data 
collection. Field managers conducted the school validations. No validation calls were made to transfer 
schools because the impact of the survey was so minimal due to most having only one child enrolled. 
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Table 4-22. Results of parent interview validations: School year 2003-04 



Parent validation status 


March 


April 


May 


June 


Number 


Percent 


Number 


Percent 


Number 


Percent 


Number 


Percent 


Completed parent interviews 


5,218 


100.0 


8,468 


100.1 


10,204 


100.0 


10,930 


100.0 


Validation cases generated 


603 


11.6 


931 


11.0 


1,140 


11.2 


1,213 


11.1 


Validation cases receipted 


258 


42.8 


632 


67.9 


864 


75.8 


1,028 


84.7 


No changes 


228 


88.4 


586 


92.7 


793 


91.8 


943 


91.7 


Minor changes 


8 


3.1 


14 


2.2 


22 


2.5 


24 


2.3 


Major changes 


22 


8.5 


32 


5.1 


49 


5.7 


61 


5.9 


Other (specify) 


0 


0.0 


0 


0.0 


0 


0.0 


0 


0.0 


Cases pending 


345 


57.2 


299 


32.1 


276 


24.2 


185 


18.3 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 



The first original school that each team completed was called to ascertain how well the 
preassessment and assessment activities went. If the feedback from the school was positive, the fifth 
school that each team completed was called. If any problems were indicated in the first validation call, 
immediate action was taken with the field supervisor. The validation feedback was discussed with the 
supervisor and remedial action was taken, including telephone training on areas needing improvement or 
in-person observation of the supervisor’s next school, if necessary; no remedial action was warranted 
based on the school validation calls. 

Field managers used a standardized telephone script, the School Validation Script, to call the 
school principals. The script covered the following topics: 

■ An overall rating of how the assessments went; 

■ Feedback the principal had received about the study from children and teachers; 

■ Suggestions for improving procedures and making it easier for a school to participate; 
and 

■ General comments and suggestions. 

Field managers called schools throughout the field period as scheduled assessments were 
completed. By the end of the field period in June, field managers had called 162 original sample schools 
(100 percent of the schools to be validated) to ascertain whether the conduct of the child assessments in 
the school was satisfactory; one school administrator could not be reached before the end of the school 
year. Table 4-23 presents the results of the school validation calls. 

Table 4-23. Results of school validation calls: School year 2003-04 

March April May June 



Number Percent Number Percent Number Percent Number Percent 



Total validation cases generated 


162 


100.0 


162 


100.0 


162 


100.0 


162 


100.0 


Validation cases completed 


28 


17.3 


109 


67.3 


156 


96.3 


162 


100.0 


Overall rating 


















Very satisfactory 


28 


100.0 


106 


97.2 


146 


93.6 


152 


93.8 


Satisfactory 


2 


7.1 


3 


2.8 


10 


6.4 


10 


6.2 


Unsatisfactory 


0 


0.0 


0 


0.0 


0 


0.0 


0 


0.0 


Very unsatisfactory 


0 


0.0 


0 


0.0 


0 


0.0 


0 


0.0 


Validation cases not completed 


134 


82.7 


53 


32.7 


6 


3.7 


1 


0.6 



NOTE: Detail may not sum to totals due to rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 
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5. DATA PREPARATION AND EDITING 



As described in chapter 4, two types of data collection instruments were used for the Early 
Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K) data collection in the spring- 
fifth grade: computer-assisted instruments and self-administered paper forms (hard copy). The data 
preparation approach differed with the mode of data collection. The direct child assessments and parent 
interview were conducted using computer-assisted interviewing (CAI) techniques. Editing specifications 
were built into the computer programs used by assessors to collect these data. The teacher, school 
administrator, and student records abstract forms were self-administered. When forms and questionnaires 
were received at Westat, coders reviewed them to ensure data readability for transfer into an electronic 
format and for the completion of critical items. The visual review included changing (upcoding) any 
“Other, specify” responses that actually fit within the available response categories of the question. For 
example, if the parent said “skiing” in response to the question on the types of exercise or physical 
activity the child participated in, this answer was upcoded to the existing category, “individual sports.” 
There were some items for which upcoding was conducted after the data were keyed due to the large 
volume of “Other” responses. Once they finished this review, the coders sent the instruments to data entry 
to be manually transferred to an electronic format and reviewed for range and logic consistency. The 
following sections describe the data preparation activities for both modes of data collection in more detail. 



5.1 Coding and Editing Specifications for Computer-Assisted Interviews (CAI) 

The very nature of designing a computer-assisted interview requires decisions about edit 
specifications to be made at the development stage. Both acceptable ranges and logical consistency 
checks were preprogrammed into the electronic questionnaires. The next few sections describe the coding 
and editing of the data that were conducted during and after the CAI parent interview. 



5.1.1 Range Specifications 

Within the CAI parent interview instruments, respondent answers were subjected to both 
“hard” and “soft” range edits during the interviewing process. A “soft range” is one that represents the 
reasonable expected range of values but does not include all possible values. Responses outside the soft 
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range were confirmed with the respondent and entered a second time. For example, the number of hours 
each week a child attended a day care center on a regular basis had a soft range of 1 to 50. A value outside 
this range could be entered and confirmed as correct by the interviewer as long as it was within the hard 
range of values (1 to 70). 

“Hard ranges” have a finite set of parameters for the values that can be entered into the 
computer, for example, “0-5 times” for the number of times the child, in the previous 5 days, ate a 
breakfast that was not provided by the school. Out-of-range values for closed-ended questions were not 
accepted. If the respondent insisted that a response outside the hard range was correct, the interviewer 
could enter the response in a comments data file. Data preparation and project staff reviewed these 
comments. Out-of-range values were accepted and entered into the data file if the comments supported 
the response. 



The child assessments did not employ hard and soft ranges. Children’s answers were 
recorded verbatim. 



5.1.2 Logical Consistency Checks (Logical Edits) 

Logical consistency checks, or logical edits, examined the relationship between responses to 
ensure that they did not conflict with one another or that the response to one item did not make the 
response to another item unlikely. For example, in the household roster, one could not be recorded as a 
mother and male. A male mother or female father (or any other gender/relationship discrepancy) would 
cause the case to fail the parent edits and the case would be examined by hand. We certainly identified 
some same-sex partnership households this way, usually by looking at the combination of relationship, 
sex, and first name. Such households would be left as is. Only cases that were deemed in error were 
corrected. When a logical error such as this occurred during an interviewing session, the interviewer was 
presented with a message requesting verification of the last response and a resolution of the discrepancy. 
In some instances, if the verified response still resulted in a logical error, the interviewer recorded the 
problem either in a comment field or on a problem report. Consistency checks were not applicable to the 
child assessments. 
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5.1.3 



Coding 



Additional coding was required for some of the items collected in the CAI instruments by 
data preparation and project staff after an interview was completed. These items included “Other, specify” 
text responses, occupation, and race/ethnicity. Interviewers entered verbatim responses to these items. 
Data preparation staff were trained to code these data using coding manuals designed by Westat and the 
National Center for Education Statistics (NCES) to support the coding process. This section describes the 
coding activities for the CAI instruments. 

Review of “Other, Specify” Items. There were twenty-five “Other, specify” open-ended 
responses in the parent interview. All of these items were reviewed to determine if they should be coded 
into one of the existing response categories. During data collection, when a respondent selected an 
“other” response in the parent interview, the interviewer entered the text into a “specify” overlay that 
appeared on the screen. The data preparation staff reviewed these text “specify” responses and, where 
appropriate, coded them into one of the existing response categories. If a response did not fit into one of 
the existing categories, it remained in “other.” If there were numerous responses that were essentially the 
same, approximately one hundred, then a new code was added to the item. Three new codes were added 
to parent interview items during fifth-grade data processing as presented in table 5-1. 



Table 5-1. “Other, specify” codes added during fifth grade: School year 2003-04 



Item 


“Other, specify” item 
descriptions 


New code 


EMQ.070 


Job Find Effort 


Read Want Ads/Intemet 


CHQ.365 


Emotional Diagnosis 


No Problem 


CHQ.337 


Behavior Diagnosis 


No Problem 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood 
Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 



The parent “Other, specify” coding system was revised from previous rounds of data 
collection and in production testing in April 2004. The revisions consisted of adding new “Other, specify” 
items that had not been part of the previous rounds; six items in the parent interview included an “Other, 
specify” code and were added to the system (see table 5-2 for new items). A total of 3,601 “Other, 
specify” text strings were processed through the parent “Other, specify” coding system. All possible 
upcodes were applied to the 10,901 cases that had at least one “Other, specify” text string. As noted 
above, whenever appropriate, responses were upcoded to existing categories. There were no “Other, 
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specify” items in the child assessments. Table 5-3 presents the number of text strings for each “Other, 
specify” item including the new ones added in fifth grade. 



Table 5-2. “Other, specify” items added to the coding system: School year 2003-04 



New item 


Description of “Other, specify” item description 




COQ.005 


Country of biological mother’s birth 




COQ.020 


Country of biological father’s birth 




CHQ.547 


Reason child no longer participates in special education 


services 


CHQ.751 


Other medication name 




CHQ.781 


Main reason for receiving family therapy 




CHQ.791 


Type of other person family saw 




SOURCE: U.S. 


Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, 


Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 




Table 5-3. 


Number of text strings by “Other, specify” item, fifth-grade parent interview: 
School year 2003-04 


Item 


“Other, specify” item description Number of strings 


Total 




3,601 


CHQ.337 


Behavior Diagnosis 


68 


COQ020 


Birth Place DAD 


13 


COQ005 


Birth Place MOM 


13 


NRQ.261 


Child Support Agreement 


82 


CCQ.393 


Center Pay Unit 


10 


EMQ.080 


Doing Last Week 


261 


CHQ.365 


Emotional Diagnosis 


160 


CHQ.726 


Exercise Types 


616 


FSQ240 


FSQ Birth Place 


4 


CMQ.690 


Interview Language 


39 


EMQ.070 


Job Find Effort 


124 


CHQ.060 


Mental Diagnosis 


159 


FSQ.181 


Nonrelative 


129 


CCQ.225 


Nonrelative Pay Unit 


1 


WPQ.130 


Other, specify Stamps 


7 


WPQ.106 


Other, specify TANF 


7 


CHQ750 


Prescription Medicine 


80 


FSQ.198 


Race 


131 


CHQ 781 


Main Reason for Receiving Family Therapy 


41 


FSQ.015 


Reason Left Household 


530 


CHQ547 


Reason No Service 


117 


CCQ.128 


Relative Pay Unit 


3 


CHQ791 


Type of Therapist Seen 


11 


CHQ. 120 


What Act Diagnosis 


54 


PIQ.015 


Why Call School 


941 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal 
Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 
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Parent Occupation Coding. As in previous data collections, occupations were coded using 
the Manual for Coding Industries and Occupations (U.S. Department of Education 1999). This coding 
manual was created for the Adult Education Survey of the National Elousehold Education Surveys 
Program (NEIES) and used an aggregated version of industry and occupation codes. The industry and 
occupation codes used by NEIES were originally developed for the 1989-90 National Postsecondary 
Student Aid Study (NPSAS) and contained one to four digits. Analysis of the NPSAS categories revealed 
that some categories had very small numbers of cases and similar categories had similar participation 
rates, suggesting that the separate codes could be collapsed without significant loss of information. The 
NEIES industry and occupation code categories use a two-digit code, the highest level of aggregation, to 
have sufficient numbers of cases to support analysis without collapsing categories. There are 13 industry 
codes and 22 occupation codes in the NEIES coding scheme. If an industry or occupation could not be 
coded using this manual, the Index of Industries and Occupations — 1980 (U.S. Department of Commerce 
1982) and Standard Occupational Classification Manual — 1980 (U.S. Department of Commerce 1980) 
were used. Both of these manuals use an expanded coding system and at the same time are directly related 
to the much more condensed NHES coding scheme. These manuals were used for reference in cases 
where the NHES manual did not adequately cover a particular situation. Exhibit 5-1 describes the 
aggregated categories that were used for coding occupation in the ECLS-K. 



Exhibit 5-1. Aggregated occupation coding categories in the ECLS-K: School years 1998-99, 2001-02, 
and 2003-04 



1 . Executive, 

Administrative, and 

Managerial 

Occupations 


This category includes senior-level and middle management occupations 
and occupations that directly support management. Senior-level managers 
are persons concerned with policymaking, planning, staffing, directing, 
and/or controlling activities. Middle managers include persons who plan, 
organize, or direct and/or control activities at the operational level. 

Workers in this category are not directly concerned with the fabrication of 
products or with the provision of services. Other officials and 
administrators include consultants, library directors, custom house builders, 
and location managers. Legislators are also included in this category. 


2. Engineers, Surveyors, 
and Architects 


The category includes occupations concerned with applying principles of 
architecture and engineering in the design and construction of buildings, 
equipment and processing systems, highways and roads, and land utilization. 


3. Natural Scientists and 
Mathematicians 


This category includes those engaged primarily in the application of 
scientific principles to research and development. Natural scientists are 
those in the physical sciences (e.g., chemistry, physics) and the life 
sciences (e.g., biology, agriculture, medicine). In addition, this category 
includes those in computer science, mathematics (including statistics), and 
operations research. 



See note at end of exhibit. 
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Exhibit 5-1. Aggregated occupation coding categories in the ECLS-K: School years 1998-99, 2001-02, 
and 2003-04 — Continued 



4. Social Scientists, 
Social Workers, 
Religious Workers, 
and Lawyers 


This category includes occupations concerned with the social needs of 
people and in basic and applied research in the social sciences. 


5. Teachers: College, 
University, and Other 
Postsecondary 
Institution; Counselors, 
Librarians, and 
Archivists 


This category includes those who teach at higher education institutions and 
at other postsecondary (after high school) institutions, such as vocational 
institutes. In addition, vocational and educational counselors, librarians, 
and archivists are included here. 


6. Teachers, except 
Postsecondary 
Institution 


This category includes prekindergarten and kindergarten teachers, 
elementary and secondary teachers, special education teachers, 
instructional coordinators, and adult education teachers (outside 
postsecondary). 


7. Physicians, Dentists, 
and Veterinarians 


This category includes health care professionals who diagnose and treat 
patients. In addition to physicians, dentists, and veterinarians, this category 
includes optometrists, podiatrists, and other diagnosing and treating 
professionals, such as optometrists, podiatrists, chiropractors, 
hypnotherapists, and acupuncturists. 


8. Registered Nurses, 
Pharmacists, 
Dieticians, Therapists, 
and Physician’s 
Assistants 


This category includes occupations concerned with the maintenance of 
health, the prevention of illness and the care of the ill through the provision 
and supervision of nursing care; compounding drugs, planning food service 
or nutritional programs; providing assistance to physicians; and the 
provision of therapy and treatment as directed by physicians 


9. Writers, Artists, 
Entertainers, and 
Athletes 


This category includes occupations concerned with creating and executing 
artistic works in a personally interpreted manner by painting, sculpturing, 
drawing, engraving, etching, and other methods; creating designs for 
products and interior decorations; designing and illustrating books, 
magazines, and other publications; writing; still, motion picture and 
television photography/filming; producing, directing, staging, acting, 
dancing, singing in entertainment; and participating in sports and athletics 
as a competitor or player and administering and directing athletic 
programs. 


10. Health Technologists 
and Technicians 


This category includes occupations concerned with providing technical 
assistance in the provision of health care. For example, clinical laboratory 
technologists and technicians, dental hygienists, radiologic technicians, 
licensed practical nurses (LPNs), and other health technologists are 
included here 


11. Technologists and 
Technicians, except 
Health 


This category includes those providing technical assistance in engineering 
and scientific research, development, testing, and related activities, as well 
as operating and programming technical equipment and systems. 


12. Marketing and Sales 
Occupations 


This category includes occupations involving selling goods or services, 
purchasing commodities and property for resale, and conducting wholesale 
or retail business. 



See note at end of exhibit. 
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Exhibit 5-1. Aggregated occupation coding categories in the ECLS-K: School years 1998-99, 2001-02, 
and 2003-04 — Continued 



13. Administrative 

Support Occupations, 
including Clerks 


This category includes occupations involving preparing, transcribing, 
transferring, systematizing, and preserving written communications and 
records; collecting accounts; gathering and distributing information; 
operating office machines and data processing equipment; operating 
switchboards; distributing mail and messages; and other support and 
clerical duties such as hank teller, data entry keyer, etc 


14. Service Occupations 


The category includes occupations providing personal and protective 
services to individuals, and current maintenance and cleaning for building 
and residences. Some examples include food service, health service (e.g., 
aides or assistants), cleaning services other than household, and personal 
services. 


15. Agricultural, Forestry, 
and Fishing 
Occupations 


This category is concerned with the production, propagation 
(breeding/growing), gathering, and catching of animals, animal products, 
and plant products (timber, crop, and ornamental); the provision of services 
associated with agricultural production; and game farms, fisheries, and 
wildlife conservation. “Other agricultural and related occupations” include 
occupations concerned with the production and propagation of animals, 
animal products, plants, and products (crops and ornamental). 


16. Mechanics and 
Repairers 


Mechanics and repairers are persons who do adjustment, maintenance, part 
replacement, and repair of tools, equipment, and machines. Installation 
may be included if installation is usually done in conjunction with other 
duties of the repairers. 


17. Construction and 
Extractive 
Occupations 


This category includes occupations that normally are performed at a 
specific site, which will change over time, in contrast to production 
workers, where the work is usually at a fixed location. Construction 
workers include those in overall construction, brickmasons, stonemasons, 
carpenters, electricians, drywall installers, paperhangers and painters, etc. 
Extractive occupations include oil well drillers, mining machine operators, 
and so on. 


18. Precision Production 
Occupations 


Precision production includes occupations concerned with performing 
production tasks that require a high degree of precision or attainment of 
rigid specification and operating plants or large systems. Examples are tool 
and die makers, pattern and model makers, machinists, jewelers, engravers, 
and so on. Also included are some food-related occupations including 
butchers and bakers. Plant and system operators include water and sewage, 
gas, power, chemical, petroleum, and other plant or system operators. 


19. Production Working 
Occupations 


This category includes occupations concerned with setting up, operating, 
and tending of machines and hand production work usually in a factory or 
other fixed place of business. 


20. Transportation and 
Material Moving 
Occupations 


This category includes occupations concerned with operating and 
controlling equipment used to facilitate the movement of people or 
materials and the supervising of those workers. 



See note at end of exhibit. 
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Exhibit 5-1. Aggregated occupation coding categories in the ECLS-K: School years 1998-99, 2001-02, 
and 2003-04 — Continued 



21. Handlers, Equipment 
Cleaners, Helpers, and 
Laborers 


This category includes occupations that involve helping other workers and 
performing routine nonmachine tasks. A wide variety of helpers, handlers, 
etc., are included in this category. Examples include construction laborers, 
freight, stock, and material movers, garage and service station related 
occupations, parking lot attendants, and vehicles washers and equipment 
cleaners. 


22. Unemployed, Retired, 
Disabled, or 
Unclassified Workers 


This category includes persons who are unemployed, have retired from the 
work force, or are disabled. It also includes unclassified occupations that 
do not fit in the categories above (e.g., occupations that are strictly 
military, such as “tank crew member” and “infantryman”). 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



Occupation coding began with an autocoding procedure using a computer string match 
program developed for the NHES. The program searched the responses for strings of text for each 
record/case and assigned an appropriate code. A little over a third of the cases were autocoded (39 
percent). Cases that could not be coded using the autocoding system were coded manually using a 
customized coding utility program designed for coding occupations. The customized coding utility 
program brought up each case for coders to assign the most appropriate codes. In addition to the text 
strings, other information, such as main duties, highest level of education, and name of the employer, was 
available for the coders. The coders used this information to ensure that the occupation code assigned to 
each case was appropriate. 

After the cases were coded (either manually or via autocoding), they were reviewed and 
verified. One hundred percent of the cases were verified. Verification of coding is an important tool for 
ensuring quality control and extending coder training. As a verification step, a second coder 
independently assigned codes (i.e., double-blind coding) to industry and occupation cases that had been 
initially coded either by the autocoding system or manually by a coder. A coding supervisor adjudicated 
disagreements between the initial code and the verification code. In the early stages, 1 00 percent of each 
coder’s work was reviewed. Once the coder’s error rate had dropped to 1 percent or less, 10 percent of the 
coder’s work was reviewed. Fifteen percent of the cases that were autocoded required adjudication 
because the verifier disagreed with the autocoding. Thirty percent of the manually coded cases required 
adjudication because the manual coder and the verifier disagreed. 
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Table 5-4 summarizes the results of the coding and verification process for occupation 
coding. In the table, manually coded indicates that occupation was initially coded by a coder as opposed 
to using the autocoding system. Discrepancies are the count of disagreements between the autocoder and 
the verifier or between the manual coder and the verifier: the discrepant cases required adjudication. The 
percentage of times in which the coding supervisor disagreed with the coder’s (or the autocoding 
system’s) initial coding is referred to as the coder error rate. The percentage of times in which the coding 
supervisor disagreed with the verifier’s coding is referred to as the verifier error rate. The denominator 
used in calculating these error rates is the number of cases verified. The error rate for manually coded 
cases was higher for coders (20.8 percent) than for verifiers (12 percent). The autocoded cases had a 
lower error rates for both coders (10.8 percent) and verifiers (5.5 percent) compared with the manually 
coded rates. 



Table 5-4. Number and percent of occupations coded, by coding status: School year 2003-04 



Coding Status 


Number coded 


Percent 


Total 


12,691 


100.0 


Coded 


12,691 


100.0 


Autocoded 


4,920 


38.8 


Manually coded 


7,771 


61.2 


Verified 


12,691 


100.0 


Verified from autocoding 


4920 


38.8 


Verified from manual coding 


7,771 


61.2 


Discrepancies between coding and verifying 


3,015 


100.0 


Adjudicated from autocoding 


720 


23.9 


Adjudicated from manual coding 


2,295 


76.1 


Autocoding system or manual coder wrong 


2,158 


100.0 


Autocoded 


521 


24.1 


Manually coded 


1,637 


75.9 


Verifier wrong 


1,056 


100.0 


Autocoded 


238 


22.5 


Manually coded 


818 


77.5 



NOTE: Occupation was collected for up to two key persons in spring-fifth grade. It was only collected for persons who had a different job 
from spring-third grade or who had not been employed in the previous round, but were employed in spring-fifth grade. For everyone else, 
the spring-third grade data were carried forward. The numbers in this table represent occupational text strings that were coded into 
appropriate occupation categories and applied to the appropriate key person (e.g., mother or father). 

1 Total discrepancies do not equal the sum of coder wrong and verifier wrong because sometimes both coder and verifier were deemed to 
be wrong. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten 
Class of 1998-99 (ECLS-K), spring 2004. 
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5.1.4 



Editing the Household Roster in the Parent Interview 



The parent data edit system was modified from spring-third grade in April and May 2004 to 
reflect changes to the fifth-grade parent interview. The parent interview data were edited in two batches 
as the interviews were completed (see table 5-5). This was done to make the process more efficient. The 
first batch consisted of all cases received from the beginning of the round through May 20, 2004. The 
second batch consisted of cases completed from May 21 through the end of data collection. 



Table 5-5. Data editing of the parent interview household roster, fifth-grade data collection: School year 
2003-04 



Batch 

number 


Data 

extraction date 


Number of 
households 
extracted 


Percent of 
total households 


Number of 
households 
failing edits 


Percent of 
households 
extracted 


Total 


t 


10,895 


100 


435 


3.2 


1 


5/20/04 


6,004 


55.1 


190 


3.1 


2 


6/3/04 


4,897 


44.9 


168 


3.4 



f Not applicable. 

NOTE: The data in this table are household-level data, not case-level data (that is, not child-level). Detail may not sum to totals due to rounding. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



The first step in the editing process was to extract the household roster data and run the data 
edits. The second step was to apply the programmatic updates to the cases failing the edits to correct any 
errors programmatically. The third step was for an expert reviewer to manually review the cases, conduct 
as-needed discussions with NCES for resolution, and resolve and correct data errors. Six cases were lost 
from the data file after the expert review and discussion with NCES because the children had no data and 
were removed from the data file. 



Several tests were run on the household roster to look for missing or inaccurate information. 
There were essentially three general types of roster tests performed to determine which cases required 
editing. First, the relationship of an individual to the focal child was compared to the individual’s listed 
age and sex. Problems found were corrected on the basis of data from prior data collections wherever 
possible. Second, households with more than one mother or more than one father were scrutinized for 
errors. While it was possible to have more than one mother in a household, such cases warranted closer 
inspection. Corrections were made whenever clear errors and a clear resolution existed. Last, the 
relationship of an individual to both the focal child and the respondent was examined, as there were cases 



5-10 







in which the relationship of an individual to the focal child conflicted with his status as the spouse/partner 
of the reference person. For example, in a household containing a child’s grandparents but not his or her 
parents, the grandmother may be designated the “mother” figure, and the grandfather thus becomes the 
“father” (for the purposes of some questions in the interview) by virtue of his marriage to the 
grandmother. These cases were examined but left unchanged. Both the original — and correct 
(grandfather) — relationship data and the new “parent-figure” designation (father) that had been 
constructed were kept. 

Updates to the household roster were required when one or more of the edit checks described 
above failed. The data for the case would be inspected, resolved, and, if necessary, corrected. There were 
435 households, with 446 children linked to them, completed a parent interview that required editing in 
fifth grade (identified by P6EDIT=1). There were 76 cases in which the interviewer noted in the Family 
Structure section (FSQ) that they had entered a person incorrectly in the household roster. These cases 
can be identified by the flag “P6ERRFLG” on the final data files. These “error” cases may or may not 
have also been edit cases, depending on whether or not the error triggered one of the edit checks. 



5.2 Coding and Editing Specifications for Hard-Copy Questionnaires 

5.2.1 Receipt Control 

In order to monitor the more than 40,000 documents that were to be received in the fifth- 
grade year, the project-specific receipt and document control system developed in the base year was used, 
with some modifications. The electronic receipt and document control system was initially loaded or 
filled with identifying information, such as identification numbers for schools, teachers, and children; the 
identification numbers linking teachers, children, and domain; and the questionnaires that were expected 
from each school and teacher for each cooperating school in the sample. As data were collected in the 
field, field supervisors completed transmittal form s for each school to indicate which questionnaires were 
being mailed to the home office. 

Once data collection started, receipt control clerks reviewed the questionnaires sent in from 
the field for accuracy and completeness. The identification number on each form was matched against the 
identification numbers in the tracking system to verify that the appropriate number of forms for each 
school was returned. Questionnaires that matched identification numbers in the tracking system were 
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receipted in the system and assigned a batch number. All receipted questionnaires matched identification 
numbers in the tracking system. Each questionnaire type had a different batch number so that 
questionnaires of the same type would be processed together. Processing data in batches was more 
efficient because it allowed staff to process in volume one type of questionnaire rather than switching 
from one type to another. Questionnaires with the same batch numbers were then compiled into batches 
of 25 instruments. The batch sheets with all the questionnaire identification numbers were printed from 
the system; the questionnaires in the batches were verified against the batch sheets. Verified batches were 
then sent for data entry. Once the batches had completed data entry, that is, all the questionnaires in a 
given batch had been keyed, the batches, accompanied by the electronic data, were returned to the data 
preparation department. The electronic data were loaded into the editing system and edited. At each point 
in the process, a flag was set in the receipt and document control system, which indicated the status of the 
instrument and the batch to which it was assigned. These statuses were the following: 

■ Batches created; 

■ Batches printed; 

■ Batches verified; 

■ Batches sent to keypunch; 

■ Batches logged from keypunch; 

■ Cases loaded to BES (editing system); 

■ Cases cleaned (in edit system); and 

■ Cases pending (in edit system). 



5.2.2 Scan Edit Procedures 

Critical items were identified for the school administrator questionnaire and the child-level 
reading, mathematics, and science teacher questionnaires. Prior to mailing the school administrator or 
child-level teacher questionnaires to Westat, the field supervisors reviewed them to ensure that critical 
items had been completed. If the critical items were missing, field supervisors attempted to retrieve them 
and recorded the outcome, completed or refused, in green pencil in the questionnaire. Exhibit 5-2 presents 
the critical items for these questionnaires. 
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Exhibit 5-2. Grade 5 critical items for questionnaires: School year 2003-04 



Instrument 


Item 


Question number 


School administrator (SAQ) 


Grade level 
School type/control 
Race/ethnicity 

School lunch program eligibility/participation 


Question 4 
Questions 5, 6, 7 
Question 8 
Question 14 


Child-level reading teacher 
questionnaires 


Grades taught 
Race/ethnicity 
Sex 

Gifted/talented, Limited English Proficient (LEP), 
Disability, Absent 


Question 15 
Question 16 
Question 17 

Question 18 


Child-level mathematics teacher 
questionnaires 


Grades taught 
Race/ethnicity 
Sex 

Gifted/talented, LEP, Disability, Absent 


Question 5 
Question 6 
Question 7 
Question 8 


Child-level science teacher 
questionnaires 


Grades taught 
Race/ethnicity 
Sex 

Gifted/talented, LEP, Disability, Absent 


Question 4 
Question 5 
Question 6 
Question 7 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class 
of 1998-99 (ECLS-K), spring 2004. 



Prior to receipting returned questionnaires, trained clerks scanned each instrument for 
completeness and assigned a status code of “Complete, No Data Retrieval,” “Complete, Data Retrieval 
Required, Complete,” or “Complete, Data Retrieval Required, Refused” based on the results of field 
supervisors efforts. Questionnaires were then logged into the receipt and document control system and 
batched for data entry. Once questionnaires were logged in, the data were first keypunched into electronic 
format and than coded and edited. Questionnaires that contained no data due to refusal by the respondents 
were logged into the receipt and document control system as “refusal.” Table 5-6 presents data on the 
number of questionnaires receipted by week. 



5-13 







Table 5-6. Number of questionnaires receipted by week, fifth-grade data collection: School year 
2003-04 



Week 


Date 


Number 

receipted 


Cumulative number 
receipted 


Percent of total 
receipted 


Total 




45,090 


45,090 


100.0 


1 


03/08/2004 


1,158 


1,158 


2.6 


2 


03/15/2004 


1,904 


3,062 


6.8 


3 


03/22/2004 


2,768 


5,830 


12.9 


4 


03/29/2004 


3,904 


9,734 


21.6 


5 


04/05/2004 


5,387 


15,121 


33.5 


6 


04/12/2004 


2,591 


17,712 


39.3 


7 


04/19/2004 


2,575 


20,287 


45.0 


8 


04/26/2004 


2,888 


23,175 


51.4 


9 


05/03/2004 


2,801 


25,985 


57.6 


10 


05/10/2004 


2,941 


28,926 


64.2 


11 


05/17/2004 


2,365 


31,291 


69.4 


12 


05/24/2004 


2,284 


33,575 


74.5 


13 


05/31/2004 


1,868 


35,443 


78.6 


14 


06/07/2004 


2,868 


38,311 


85.0 


15 


06/14/2004 


1,368 


39,679 


88.0 


16 


06/21/2004 


3,074 


42,753 


94.8 


17 


06/28/2004 


1,357 


44,110 


97.8 


18 


07/05/2004 


882 


44,992 


99.8 


19 


07/12/2004 


98 


45,090 


100.0 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, 
Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 



The following sections describe the data entry, coding, and editing processes for hard-copy 
questionnaires. 



5.2.3 Data Entry 



Data entry consisted of two steps: (1) entering the data and (2) verifying that the data had 
been entered accurately. Westat data entry staff keyed the form s in each batch. A set of keying rules was 
established for each questionnaire and followed by the data entry staff. To verify the accuracy of the data 
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entry, more senior data entry operators then rekeyed 1 00 percent of the data. The results of the two data 
entry sessions were compared and differences identified. When differences were found, the hard-copy 
form was pulled and examined to determine what corrections, if any, had to be made to the keyed data. 
These corrections were rekeyed, resulting in an accuracy rate exceeding 99 percent. The verified batches 
were then transmitted electronically to Westat’s computer system for data editing. 



5.2.4 Coding 

The hard-copy questionnaires required a quick visual review of particular questions in each 
questionnaire, coding of race/ethnicity for teachers, and review of “Other, specify” text responses. The 
quick visual review was to ensure that the questionnaire values accurately reflected existing categories, 
were complete and consistent across variables, and that the numbers were converted to the appropriate 
unit of measurement prior to converting data to an electronic format. Once the hard-copy questionnaires 
had been visually reviewed, they were coded. The coding staff was trained on the procedures and had 
manuals to support the coding process. Senior coders verified coding. The verification rate was set at 1 00 
percent for each coder until accuracy of less than 1 percent error rate was established. After that point, 
work was reviewed at a rate of 1 0 percent. 

The “Other, specify” text responses were reviewed by the data editing staff and, where 
appropriate, upcoded into one of the existing response categories. The specify responses that remained 
after upcoding were reviewed to evaluate whether the addition of any new response categories would be 
appropriate. There was no need for the addition of new response categories in fifth grade. 



5.3 Data Editing Management Process 

The management of the data editing process involved the creation of several data files, 
including the Collection Database, Holding Database, Editing Database, and Delivery Database. 
Exhibit 5-3 provides a diagram of the process described below. 
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Exhibit 5-3. 



ECLS-K hard-copy data editing, fifth-grade data collection: School year 2003-04 



From Field 



Copy as is. Set Collection Status to 
CollectionNew 

T 



Collection 



Check New and CheckEdit cases, 
set Holding status as follows: 

- if case is DIRTY 

- if status was New, set status to 
DirtyNew. 

- if status was CheckEdit, set 
status to DirtyEdit. 

- if case is CLEAN 

- if status was New, set status to 
CleanNew. 

- if status was CheckEdit, set 
status to CleanEdit. 

Other assorted edits and 
transpose issues. 

Apply other specify upcodes. 
Holding status would be set to 
InDelivery. 



Copy to Holding and set Holding status to New. Set 
Collection status to CollectionCopied. Apply code-all- 
that-apply recoding and yes/no recoding. Extract 
Other Specify text. Convert all blanks to -9, and all 
don't knows and refused to -8 and -7 as appropriate. 



Error Files 




Editor will look at/analyze/ 
change the cases that are 
WaitingForEdit. 

After the editor is done, the 
editor will assign a status of 
WasEdited or KeepAsIs to the 
case. KeepAsIs is assigned only 
if there are rule violations which 
will not be changed. 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 






5.3.1 



Collection Database 



This database contained the keyed records for hard-copy questionnaires. One Collection 
Database was created for each instrument and as additional data were keyed, the cases were added to the 
database. The Collection Databases were Blaise databases. The ASCII file resulting from the key entry 
process was converted to Blaise data in the Collection Database so that they could be merged with the 
parent interview data and undergo additional data review (see section 5.4) 

Records in the Collection Databases were assigned status codes reflecting their current 
status. All new records were assigned a status of CollectionNew. When cases were copied to the Holding 
Database, the status was updated to CollectionCopied. The data in the Collection Database were retained 
in their original form; that is, they were not modified based upon later steps. 



5.3.2 Holding Database 

Data were copied from the Collection Database to the Holding Database for the editing 
process. The Holding Database for each instrument was also a Blaise database. The copied cases were 
assigned a status code of New. Cases that had already been involved in a prior editing cycle and had been 
returned to the Holding Database were assigned a status of CheckEdit or KeepAsIs. 

As the data were copied from the Collection Database to the Holding Database, a number of 
processes were run. Code-all-that-apply (COTA) recoding and “yes/no” recoding were applied. COTA 
recoding involved changing the multiple-response values of 0/1, 0/2, 0/3, etc., to a series of yes/no (1/2) 
responses. Yes/no recoding provided a means to resolve questions left unanswered in a series of yes/no 
items. If all marked answers were “Yes,” then the unanswered items were converted to “No.” However, if 
any item was “No,” “Don’t know,” or “Refused,” all unanswered items were converted to -9 (Not 
ascertained). All blanks were converted to -9 (Not ascertained) and don’t know and refused responses 
were converted to -7 and -8 as appropriate. It was at this stage that skip patterns were enforced using the 
Blaise CheckRuIes function and legitimate skips were assigned the standard code of-1. 

Edit programs (range and logical checks) were run against all cases contained in the Holding 
Database. As the editing process continued, the Holding Database contained both new cases copied from 
the Collection Database and edited cases returned from the Editing Database (see section 5.3.3). Each 
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case was assigned a status code that reflected its current status. For cases that were new to the Flolding 
Database, the CheckRules function assigned one of two codes. The status CleanNew was assigned to new 
cases that contained no edit (range or logical) errors. The status DirtyNew was assigned to new cases that 
failed one or more edit checks. Those cases that had undergone edit updating were also subjected to edit 
checks to identify any errors that remained or were inadvertently introduced during edit updating. The 
CheckRules function assigned the status CleanEdit to cases with no remaining errors. The status DirtyEdit 
was assigned to cases returned from edit updating that had remaining or new errors. Those cases that were 
assigned a status of KeepAsIs in a previous editing round were considered clean. 

Cases that were found to have edit errors (DirtyNew and DirtyEdit) were copied to the 
Editing Database for review and updating. At that time, their status in the Flolding Database was set to 
InEdit. A face sheet was generated for each case with editing errors, giving the batch number, case ID, 
and edit rules that had been violated. 



5.3.3 Editing Database 

Cases in the Flolding Database that failed edit checks were copied to the Editing Database 
for the correction of errors. As cases were copied to the Editing Database, they were assigned a status of 
WaitingForEdit in the Editing Database. Editing staff worked from face sheets produced during the edit 
checks conducted on the Flolding Database to retrieve and correct case records. Using the batch number 
and case ID number, editors retrieved and reviewed hard-copy instruments as necessary to resolve editing 
errors. 



Once the editor had reviewed and updated each case as necessary, he or she assigned one of 
two outcome codes. The status code of WasEdited was assigned when all edit errors had been corrected. 
A status of KeepAsIs was assigned when the editor’s review indicated that data that violated an edit check 
should be retained, for example, when the hard-copy instrument indicated that an out-of-range value was 
correct. 



Cases with the statuses of WasEdited and KeepAsIs were moved back to the Holding 
Database. Cases that had a status of WasEdited in the Editing Database were assigned the status 
CheckEdit in the Holding Database. The edit rules were applied to these cases to ensure that they were 
clean. As noted earlier, cases assigned a status of KeepAsIs in the editing process were considered clean. 
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5.3.4 



Delivery Database 



The main purpose of the Delivery Database was to store the instrument data at the school, 
teacher, or child level in a “rectangular” format consistent with downstream activities in preparation for 
data delivery. Cases for which editing and coding activities were completed were copied from the 
Holding Database to the Delivery Database. These were cases with status codes of CleanNew, CleanEdit, 
or KeepAsIs. When the data were copied to the Delivery Database, the “Other, specify” upcodes and 
parent interview occupation codes were applied. See exhibit 5-4 for a summary of the status codes 
assigned for data management databases. 

Exhibit 5-4. Status codes assigned for data management databases, fifth-grade data collection: School 
year 2003-04 

Database Status codes assigned 

Collection CollectionNew : New cases 

CollectionCopied: Copied to Holding Database 

Holding New: New case copied from the Collection Database 

CheckEdit: Cases returned from the Editing Database to be subjected to edit checks again 
DirtyNew: New cases that fail edit check(s) 

DirtyEdit : Cases returned to from the Editing Database that have been edited and have 
failed edit check(s) 

CleanNew: New cases with no errors 
CleanEdit: Edited cases with no errors 
InEdit: Cases copied to the Editing Database 

KeepAsIs: Cases returned from the Editing Database that have been reviewed in the 
Editing Database and are not to be subjected to edits again 

Editing WaitingForEdit: Cases with errors waiting to be edited 

WasEdited: Cases for which editing is completed 

KeepAsIs: Edited cases for which the editor has determined that edit errors (e.g., out of 
range responses) should be retained as is 

Delivery InDelivery: Cases ready for delivery 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 
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5.4 



Data Editing 



The data editing process consisted of running range edits for soft and hard ranges, running 
consistency edits in Blaise, and then manually reviewing frequencies of the results. 



5.4.1 Range Specifications 

Hard-copy range specifications set the parameters for high and low acceptable values for a 
question. Where values were printed on the forms, these were used as the range parameters. For open- 
ended questions, such as, “Counting this school year, how many years have you taught in your current 
school including part-time teaching?” high and low ranges were established as acceptable values. Data 
frequencies were run on the range of values to identify any errors. Values outside the range were 
identified as errors and were printed for a data editor to review. Cases with range errors were identified, 
and the original response was updated. In some cases, range violations were retained in the data because 
the value was checked and found to be the value reported by the teacher or school. These were marked as 
KeepAsIs cases. Data frequencies were then rerun and reviewed. This iterative process was repeated until 
no further range errors were found. 



5.4.2 Consistency Checks (Logical Edits) 

By programming logical edits between variables, consistency between variables not involved 
in a skip pattern was confirmed. For example, in the school administrator questionnaire, the number of 
children eligible for free breakfast could not exceed the total number of children enrolled in the school. 
These logical edits were run on the whole database after all data entry and range edits were complete. The 
logical edits were run separately for each form. All batches of data were combined into one large data file, 
and data frequencies were produced. The frequencies were reviewed to ensure the data remained logically 
consistent within the form. When an inconsistency was found, the case was identified and the 
inconsistency was printed for an editor to review. The original value was corrected (or checked and “kept 
as is” if the data item was confirmed, and the case was again run through the consistency edits. Once the 
case passed the consistency edits, it was returned to the main data set. The frequencies were then rerun 
and reviewed. This was an iterative process; it was repeated until no further inconsistencies were found. 
Table 5-7 shows hard-copy questionnaire data preparation production. More than 87 percent of all 
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questionnaires passed all the edits. The student record abstract accounted for 83 percent of all cases 
failing edits and, for all but a handful of cases, the data remained as it was reported. 



Table 5-7. Hard-copy editing progress report, fifth-grade data collection: School year 2003-04 



Instrument type 


Number of 
instruments 


Number of 
instruments 
clean after the 
first cleaning 
pass 


Number of 
instruments 
edited 


Instruments edited 

Number of Number of 

instruments clean instruments left 
from edit as is from edit 


Total 


43,518 


37,908 


5,610 


357 


5,253 


Percent 




87.1 


12.9 


0.82 


12.1 


Facilities checklist 


2,183 


2,183 


0 


0 


0 


School administrator 
questionnaire 


2,123 


1,826 


297 


88 


209 


Special education A 


722 


712 


10 


6 


4 


Special education B 


981 


888 


93 


33 


60 


Student records 
abstract 


10,122 


5,461 


4,661 


3 


4,658 


Teacher 

questionnaire 


5,683 


5,530 


153 


91 


62 


Reading 

questionnaire 


10,878 


10,655 


223 


109 


114 


Math questionnaire 


5,381 


5,299 


82 


18 


64 


Science questionnaire 


5,445 


5,354 


91 


9 


82 



NOTE: The total in this table does not match the total receipted shown in table 5-6 because of refusals and other unusable 
questionnaires. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



5.4.3 Teacher Responses to Key Child Items 

In fifth grade, teachers of sampled children were asked to respond to child-level 
questionnaires for the reading, mathematics, and science domains. In many cases, teachers had more than 
one sampled child in a class. The items in the child-level questionnaire that collected information about 
classroom characteristics were redundant under these circumstances. The key child approach was 
designed to minimize the burden on the teachers by designating one questionnaire in which the classroom 
characteristics items were to be completed. See section 4.4.2 for a description of the key child design and 
procedures. 
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Once the child-level questionnaires were keyed and loaded into the editing system, a review 
was conducted to identify cases in which teachers reported classroom characteristics on a different 
questionnaire than the one designated as the key child instrument for the given class. This process 
involved three steps: the review of missing data for classroom characteristics items within each domain 
(reading, mathematics, and science) for key child records, a detailed review of all data records in classes 
with multiple children and missing values for selected classroom characteristics items, and the updating 
of appropriate records. 

In the first step, data records for key children in all classrooms with more than one sampled 
child were selected. Frequency distributions of the classroom items were examined for the level of 
missing data within each domain. All classroom characteristics items were included in this review. The 
results of this initial review indicated that missingness was largely confined to the items concerning the 
race and sex composition of the classroom. 

In the second step, all returned instruments were selected for classrooms with multiple 
children that had missing data for the race and sex composition items. These cases were reviewed to 
ascertain whether the teacher had mistakenly reported the classroom characteristics items on a 
questionnaire other than that designated for the key child. 

In the third step, update specifications were prepared, directing data preparation staff to 
apply the classroom characteristics data to the key child record for the classroom. Updates were made to 
10 reading records, 5 mathematics records, and 3 science records as a result of this review. 

A review was also conducted to identify classrooms with multiple sampled children for 
which no key child instrument was returned. There were 5 such cases for reading, 7 such cases for 
mathematics, and 3 such cases for science. Another child for whom an instrument was returned was 
designated as the key child in these classrooms. 



5.4.4 Frequency and Cross-Tabulation Review 

As a final review, frequencies and cross-tabulations were run to determine consistency and 
accuracy within the various forms. If discrepancies could not be explained, no changes were made to the 
data. For example, in teacher questionnaire A, an item asking about languages other than English spoken 
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in the classroom included a response option of “No language other than English.” If a respondent circled 
that response, but also answered (in subsequent items) that other languages besides English were spoken 
in the classroom, then the response was left as recorded by the respondent because the discrepancy could 
not be resolved. 



5.5 Creation of the Socioeconomic Status (SES) Variable 

Socioeconomic status (SES) was computed at the household level using data for the set of 
parents who completed the parent interview in fifth grade. The SES variable reflects the socioeconomic 
status of the household at the time of data collection for fifth grade (spring 2004). The components used 
to create the SES were as follows: 

1. Father/male guardian’s education; 

2. Mother/female guardian’s education; 

3. Father/male guardian’s occupation; 

4. Mother/female guardian’s occupation; and 

5. Household income. 

Occupation was recoded to reflect the average of the 1989 General Social Survey (GSS) 
prestige score. This was computed as the average of the corresponding prestige scores for the 1980 
Census occupational categories covered by the ECLS-K occupation. Table 5-8 provides details of the 
recode of the occupation values to the 1989 GSS prestige score values 

Not all parents completed the parent interview; among those who did, not all responded to 
every question. Therefore, there were missing values for some of the components of the SES composite 
variable. For a description of how data for the components of the SES were collected, see section 7.5.2. 7 
of the Combined User’s Manual for the ECLS-K Fifth-Grade Data Files and Electronic Codebooks 
(NCES 2006-032) (Tourangeau et al. forthcoming). Table 5-9 shows that only a small percentage of 
values for the education and occupation variables were missing; a much larger proportion of households 
had missing values for the detailed income range. The total number of households in the third-grade data 
file (identified by the parents in the households) is 10,895, of which 77.8 percent are households with 
both parents present. Of the households with single parents, 10.6 percent are headed by single fathers. 
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Table 5-9 also includes labor force status that was used to determine whether missing occupation data 
would be imputed. Individuals were defined as “in the labor force” if they were working at a paid job, on 
vacation from a paid job, or looking for a job. Occupation was imputed only for parents in the labor force. 
A description of the levels of the SES components can be found in table 5-10. 



Table 5-8. Recode of occupation values to 1989 GSS prestige scores, spring-fifth grade: School 
year 2003-04 



ECLS-K occupation code 


1980 Census 
occupation code 


1989 GSS 
prestige score 


01: Executive, administration, managerial 


003-037 


53.50 


02: Engineer, surveyor, architect 


043-059 


64.89 


03: Natural scientist, mathematician 


063-083 


62.87 


04: Social scientist, social worker, lawyer 


166,168-179 


59.00 


05: University teacher, postsecondary counselor, 
librarian 


113-149,163-165 


72.10 


06: Teacher, except postsecondary 


153-159 


63.43 


07 : Physician, dentist, veterinarian 


084-089,167 


77.50 


08: Registered nurse, pharmacist 


095-106 


61.56 


09: Writer, artist, entertainer, athlete 


183-199 


52.54 


10: Health technologist/technician 


203-208 


57.83 


1 1 : Other technologist/technician 


213-235 


48.69 


12: Marketing/sales occupation 


243-285 


35.78 


13: Administrative support 


303-389 


38.18 


14: Service occupation 


403-427,433-469 


34.95 


15: Agriculture, forestry, fishing occupation 


473-499 


35.63 


16: Mechanics/repairs 


503-549 


39.18 


17: Construction/Extractive occupation 


553-599 


39.20 


18: Precision production occupation 


613-699 


37.67 


19: Other production occupation 


703-799 


33.42 


20: Transportation, material moving 


803-859 


35.92 


21: Handler, cleaner, helper, laborer 


863-889 


29.60 


22: Unemployed/retired 




Missing 


No occupation 




Missing 


Cannot be coded 




Missing 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten 
Class of 1998-99 (ECLS-K), spring 2004. 
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Table 5-9. Parents or households with missing value of the SES components, spring- 
fifth grade: School year 2003-04 



SES component 


Number missing 


Percent missing 


Education 


Father 


160 


1.8 


Mother 


159 


1.5 


Labor force status 


Father 


88 


1.0 


Mother 


106 


1.0 


Occupation 1 


Father 


166 


1.9 


Mother 


100 


0.9 


Detailed income range 3 


Total 


883 


8.1 


Broad income range 4 known 


472 


4.3 


Broad income range unknown 


411 


3.8 



1 Percent missing is over the number of records that should have nonmissing value of the imputed variable. Note that not 
applicable cases are not included in denominator. 

2 These numbers include parents with missing labor force status. 

3 Detailed income range: $5,000 or less, $5,001 -$10,000, $10,001 -$15,000, $15,001 -$20,000, $20,001 -$25,000, $25,001 -$30,000, 
$30,001 -$35,000, $35,001 -$40,000, $40,001 -$50,000, $50,001-$75,000, $75,001 -$100,000, $100,001 -$200,000, more than 
$ 200 , 000 . 

4 Broad income range: $25,000 or less, $25,001 or more. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, 
Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 
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Table 5-10. Components of the SES variable, spring-fifth grade: School year 2003-04 



Component 


Level 


Description 


Education level of 


1 


8th grade or below 


mother/father/legal guardian 


2 


9th to 12th grade 




3 


High school diploma or equivalent 




4 


Vocational/technical program 




5 


Some college 




6 


Bachelor’s degree 




7 


Graduate or professional school, no degree 




8 


Master’s degree 




9 


Doctorate or professional degree 


Occupation of 


1 


Executive, administration, managerial 


mother/father/legal guardian 


2 


Engineer, surveyor, architect 




3 


Natural scientist, mathematician 




4 


Social scientist, social worker, lawyer 




5 


University teacher, postsecondary counselor, librarian 




6 


Teacher, except postsecondary 




7 


Physician, dentist, veterinarian 




8 


Registered nurse, pharmacist 




9 


Writer, artist, entertainer, athlete 




10 


Health technologist/technician 




11 


Other technologist/technician 




12 


Marketing/sales occupation 




13 


Administrative support 




14 


Seivice occupation 




15 


Agriculture, forestry, fishing occupation 




16 


Mechanics/repairs 




17 


Construction/extractive occupation 




18 


Precision production occupation 




19 


Other production occupation 




20 


Transportation, material moving 




21 


Handler, cleaner, helper, laborer 




22 


Unemployed/retired 




97 


No occupation 




98 


Cannot be coded 


Income broad range 


1 


$25,000 or less 




2 


More than $25,000 


Income detailed range 1 


2,500 


$5,000 or less 




7,500 


$5,001 to $10,000 




12,500 


$10,001 to $15,000 




17,500 


$15,001 to $20,000 




22,500 


$20,001 to $25,000 




27,500 


$25,001 to $30,000 




32,500 


$30,001 to $35,000 




37,500 


$35,001 to $40,000 




45,000 


$40,001 to $50,000 




62,500 


$50,001 to $75,000 




87,500 


$75,001 to $100,000 




150,000 


$100,001 to $200,000 




300,000 


More than $200,000 



1 The midpoints of the detailed ranges are the levels used for this variable. For the top income range (more than $200,000), the 
level was computed based on the midpoint for the parents reporting in this range in the first-grade year. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, 
Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 
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A two-stage procedure was used to impute missing values for each component of the SES 
composite variable. First, if a parent had completed an interview in the kindergarten, first-grade, or third- 
grade year, missing values for the fifth-grade education, occupation, and detailed income range were 
filled in with values from the previous years. The rationale for this approach was that the best source of 
data for an individual or a household was the data from a previous year. 



This first imputation stage was implemented as follows: 

1. Education level was brought forward from the most recent round of data collection 
prior to fifth grade, but only if the fifth-grade parent was the same person for whom 
the education information was reported in that previous round. 

2. Labor force status (whether the parent was in the labor force) was brought forward 
from the most recent round of data collection prior to fifth grade, but only if the fifth- 
grade parent was the same person who reported the labor force status information 
from that previous round. Even though labor force status is not a direct component of 
the SES, it cannot be missing since it was used to determine how to impute missing 
occupation information. 

3. Occupation was brought forward from the most recent round of data collection prior 
to fifth grade, but only if the fifth-grade parent was in the labor force (i.e., he or she 
was working at a paid job, on vacation from a paid job, or looking for a job) and was 
the same person who reported the occupation information from that previous round. 

4. Detailed income category was brought forward from the most recent round of data 
collection prior to fifth grade. 

Second, data still missing after this initial step were imputed using a hot deck methodology. 
In hot deck imputation, the value reported by a respondent for a particular item was assigned or “donated” 
to a “similar” person who failed to respond to that question. Auxiliary information known for both donors 
and nonrespondents was used to form groups of persons having similar characteristics. These groups of 
similar respondents and nonrespondents are called “imputation cells.” The imputed value for a case with a 
missing value is taken from a randomly selected donor among the respondents within the cell. 



Imputation cells were defined by respondent characteristics that were the best predictors of 
the variables to be imputed. These relationships had been determined previously by CHAID (Chi-squared 
Automatic Interaction Detector) analyses of the base year data, as shown in table 5-11. Missing values for 
the education, occupation, and detailed income range variables were imputed by the hot deck method for 
all households. Hot deck imputation was done in a sequential order, separately, by type of household 
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Table 5-11. Demographic characteristics used in creating imputation cells, spring-fifth 
grade: School year 2003-04 



Characteristic 


Level 


Description 


Census region 


1 


Northeast 




2 


Midwest 




3 


South 




4 


West 


School affiliation 


1 


Private: Catholic 




2 


Private: Religious, non-Catholic 




3 


Private: Nonreligious 




4 


Public: Regular, Department of 
Defense, Bureau of Indian 
Affairs 


School locale 


1 


Large city 




2 


Midsize city 




3 


Urban fringe, large city 




4 


Urban fringe, midsize city 




5 


Large town 




6 


Small town 




7 


Rural, outside MSA 




8 


Rural, inside MSA 


Household type 


1 


Female single parent 




2 


Male single parent 




3 


Both parents present 


Race/ethnicity 


1 


White 




2 


Black 




3 


Hispanic 




4 


Asian 




5 


Pacific Islander 




6 


American or Alaskan Native 




7 


More than one race 



See note at end of table. 
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Table 5-11. Demographic characteristics to be used in creating imputation cells, spring- 
fifth grade: School year 2003-04 — Continued 



Characteristic 


Level 


Description 


Mother’s age, 


1 


16-17 years 


father’s age 


2 


18-19 years 




3 


20-21 years 




4 


22-23 years 




5 


24-25 years 




6 


26-27 years 




7 


28-29 years 




8 


30-31 years 




9 


32-33 years 




10 


34-35 years 




11 


36-37 years 




12 


38-39 years 




13 


40-41 years 




14 


42-43 years 




15 


44-45 years 




16 


46-47 years 




17 


48-49 years 




18 


50-51 years 




19 


52-53 years 




20 


54-55 years 




21 


56-57 years 




22 


58-59 years 




23 


60-61 years 




24 


62-63 years 




25 


64-65 years 




26 


66-67 years 




27 


68-69 years 




28 


70 or more years 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal 
Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 



(female single parent, male single parent, and both parents present). For households with both parents 
present, the mother’s and father’s variables were imputed separately. Imputed as well as reported values 
were used to define imputation cells; missing values for donor characteristics were treated as a separate 
category. No imputed value was used as a donor. No donor was used more than once. 

Occupation imputation involved two steps. First, the labor force status of the parent was 
imputed (i.e., whether the parent was employed). Then the parent’s occupation was imputed only for 
those parents whose status was identified as employed either through the parent interview or the first 
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imputation step. The detailed income range was imputed in two steps: first for cases where the broad 
income range was known, and second for cases where it was unknown. 

For households where both parents were present, the order of hot deck imputation was as 

follows: 

1. Mother’s education; 

2. Father’s education; 

3. Mother’s labor force status; 

4. Father’s labor force status; 

5. Mother’s occupation; 

6. Father’s occupation; 

7. Detailed income range, where the broad income range was known; and 

8. Detailed income range, where the broad income range was u nk nown. 

Following imputation, all values of the SES components were nonmissing including the 
value -1 for “not applicable.” Examples of not applicable cases are children with no father in the 
household, in which case the father’s education and occupation are not applicable. 

Table 5-12 summarizes the imputation results. Tables 5-13 to 5-19 summarize the 
distribution of the records before and after imputation. The percentage columns may not always add to 
100 percent due to rounding. 
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Table 5-12. Imputation of SES components by filling in with values from previous rounds or by hot 
deck, spring-fifth grade: School year 2003-04 





Number 


Filled in with values 
from previous round 


Imputed by 
hot deck 


SES component 


missing 


n 


Percent 


n 


Percent 


Education 


Father 


160 


97 


60.6 


63 


39.4 


Mother 


159 


108 


67.9 


51 


32.1 


Labor force status 


Father in labor force status 


79 


66 


83.5 


13 


16.5 


Father not in labor force status 


9 


7 


77.8 


2 


22.2 


Mother in labor force status 


75 


68 


90.7 


7 


9.3 


Mother not in labor force status 


31 


26 


83.9 


5 


16.1 


Occupation 1 


Father 


166 


16 


9.6 


150 


90.4 


Mother 


100 


18 


18.0 


82 


82.0 


Detailed income range 


Total 


883 


786 


89.0 




11.0 


Broad income range known 


472 


388 


82.2 


84 


17.8 


Broad income range unknown 


411 


398 


96.8 


13 


3.2 



1 Occupation was not imputed if parent was not in labor force (whether labor force was filled in with data from previous round or imputed 
by hot deck). 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten 
Class of 1998-99 (ECLS-K), spring 2004. 
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Table 5-13. Mother’s education, before and after imputation, spring-fifth grade: School 
year 2003-04 



Mother’s 
education level 


Before imputation 


Imputed 


After imputation 


Count 


Percent 


Count 


Percent 


Count 


Percent 


Total 


10,895 


100.0 


159 


100.0 


10,895 


100.0 


1 


447 


4.1 


21 


13.2 


468 


4.3 


2 


633 


5.8 


23 


14.5 


656 


6.0 


3 


2,622 


24.1 


44 


27.7 


2,666 


24.5 


4 


558 


5.1 


10 


6.3 


568 


5.2 


5 


3,124 


28.7 


37 


23.3 


3,161 


29.0 


6 


1,915 


17.6 


17 


10.7 


1,932 


17.7 


7 


285 


2.6 


1 


0.6 


286 


2.6 


8 


662 


6.1 


3 


1.9 


665 


6.1 


9 


233 


2.1 


3 


1.9 


236 


2.2 


Not applicable 


257 


2.4 


0 


0.0 


257 


2.4 


Missing 


159 


1.5 


t 


t 


0 


0.0 



f Not applicable. 

NOTE: Percentages may not sum to 100 due to rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, 
Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 



Table 5-14. Father’s education, before and after imputation, spring-fifth grade: School year 
2003-04 



Father’s 
education level 


Before imputation 


Imputed 


After imputation 


Count 


Percent 


Count 


Percent 


Count 


Percent 


Total 


10,895 


100.0 


160 


100.0 


10,895 


100.0 


1 


375 


3.4 


16 


10.0 


391 


3.6 


2 


595 


5.5 


24 


15.0 


619 


5.7 


3 


2,253 


20.7 


40 


25.0 


2,293 


21.1 


4 


472 


4.3 


9 


5.6 


481 


4.4 


5 


1,955 


17.9 


34 


21.3 


1,989 


18.3 


6 


1,556 


14.3 


25 


15.6 


1,581 


14.5 


7 


215 


2.0 


3 


1.9 


218 


2.0 


8 


664 


6.1 


5 


3.1 


669 


6.1 


9 


492 


4.5 


4 


2.5 


496 


4.6 


Not applicable 


2,158 


19.8 


0 


0.0 


2,158 


19.8 


Missing 


160 


1.5 


t 


t 


0 


0.0 



f Not applicable. 

NOTE: Percentages may not sum to 100 due to rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, 
Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 
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Table 5-15. Mother’s labor force status, before and after imputation, spring-fifth grade: 
School year 2003-04 



Mother’s labor 


Before imputation 


Imputed 


After imputation 


force status 


Count 


Percent 


Count 


Percent 


Count 


Percent 


Total 


10,895 


100.0 


106 


100.0 


10895 


100.0 


1 


8,202 


75.3 


75 


70.8 


8,277 


76.0 


2 


2,330 


21.4 


31 


29.3 


2,361 


21.7 


Not applicable 


257 


2.4 


0 


0.0 


257 


2.4 


Missing 


106 


1.0 


t 


t 


0 


0.0 


f Not applicable. 














NOTE: Percentages may not sum to 100 due to rounding. 










SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, 


Kindergarten Class of 1 998 


-99 (ECLS-K), spring 2004. 










Table 5-16. Father’s labor force status, before and after imputation, spring-fifth grade: 


School year 2003 


-04 










Father’s labor 


Before imputation 


Imputed 


After imputation 


force status 


Count 


Percent 


Count 


Percent 


Count 


Percent 


Total 


10,895 


100.0 


88 


100.0 


10,895 


100.0 


1 


8,254 


75.8 


79 


89.8 


8,333 


76.5 


2 


395 


3.6 


9 


10.2 


404 


3.7 


Not applicable 


2,158 


19.8 


0 


0.0 


2,158 


19.8 


Missing 


88 


0.8 


f 


f 


0 


0.0 



f Not applicable. 

NOTE: Percentages may not sum to 100 due to rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, 
Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 
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Table 5-17. Mother’s occupation, before and after imputation, spring-fifth grade: School year 
2003-04 



Mother’s occupation 


Before imputation 


Imputed 




After imputation 


Count 


Percent 


Count Percent 


Count 


Percent 


Total 


10,895 


100.0 


100 


100.0 


10,895 


100.0 


Not in labor force 


2,361 


21.7 


t 


t 


2,361 


21.7 


1 


951 


8.7 


6 


6.0 


957 


8.8 


2 


44 


0.4 


0 


0.0 


44 


0.4 


3 


65 


0.6 


1 


1.0 


66 


0.6 


4 


184 


1.7 


0 


0.0 


184 


1.7 


5 


72 


0.7 


0 


0.0 


72 


0.7 


6 


632 


5.8 


4 


4.0 


636 


5.8 


7 


70 


0.6 


0 


0.0 


70 


0.6 


8 


454 


4.2 


2 


2.0 


456 


4.2 


9 


138 


1.3 


0 


0.0 


138 


1.3 


10 


195 


1.8 


1 


1.0 


196 


1.8 


11 


130 


1.2 


2 


2.0 


132 


1.2 


12 


730 


6.7 


7 


7.0 


737 


6.8 


13 


2,137 


19.6 


28 


28.0 


2,165 


19.9 


14 


1,580 


14.5 


34 


34.0 


1,614 


14.8 


15 


74 


0.7 


1 


1.0 


75 


0.7 


16 


13 


0.1 


0 


0.0 


13 


0.1 


17 


23 


0.2 


0 


0.0 


23 


0.2 


18 


50 


0.5 


1 


1.0 


51 


0.5 


19 


430 


4.0 


8 


8.0 


438 


4.0 


20 


80 


0.7 


1 


1.0 


81 


0.7 


21 


109 


1.0 


3 


3.0 


112 


1.0 


22 


16 


0.2 


1 


1.0 


17 


0.2 


Not applicable 


257 


2.4 


0 


0.0 


257 


2.4 


Missing 


100 


0.9 


t 


t 


0 


0.0 



f Not applicable. 

NOTE: Percentages may not sum to 100 due to rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten 
Class of 1998-99 (ECLS-K), spring 2004. 
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Table 5-18. Father’s occupation, before and after imputation, spring-fifth grade: School year 
2003-04 



Father’s occupation 


Before imputation 


Imputed 




After imputation 


Count 


Percent 


Count Percent 


Count 


Percent 


Total 


10,895 


100.0 


166 


100.0 


10,895 


100.0 


Not in labor force 


404 


3.7 


t 


t 


404 


3.7 


1 


1,453 


13.3 


25 


15.1 


1,478 


13.6 


2 


303 


2.8 


3 


1.8 


306 


2.8 


3 


131 


1.2 


2 


1.2 


133 


1.2 


4 


198 


1.8 


1 


0.6 


199 


1.8 


5 


62 


0.6 


0 


0.0 


62 


0.6 


6 


147 


1.4 


0 


0.0 


147 


1.4 


7 


174 


1.6 


1 


0.6 


175 


1.6 


8 


60 


0.6 


1 


0.6 


61 


0.6 


9 


100 


0.9 


1 


0.6 


101 


0.9 


10 


48 


0.4 


1 


0.6 


49 


0.5 


11 


260 


2.4 


5 


3.0 


265 


2.4 


12 


670 


6.2 


8 


4.8 


678 


6.2 


13 


414 


3.8 


5 


3.0 


419 


3.9 


14 


669 


6.1 


19 


11.5 


688 


6.3 


15 


316 


2.9 


8 


4.8 


324 


3.0 


16 


665 


6.1 


11 


6.6 


676 


6.2 


17 


895 


8.2 


30 


18.1 


925 


8.5 


18 


133 


1.2 


3 


1.8 


136 


1.3 


19 


677 


6.2 


14 


8.4 


691 


6.3 


20 


562 


5.2 


15 


9.0 


577 


5.3 


21 


202 


1.9 


13 


7.8 


215 


2.0 


22 


28 


0.3 


0 


0.0 


28 


0.3 


Not applicable 


2,158 


19.8 


0 


0.0 


2,158 


19.8 


Missing 


166 


1.5 


t 


f 


0 


0.0 



t Not applicable. 

NOTE: Percentages may not sum to 1 00 due to rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal 
Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 
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Table 5-19. Household income, before and after imputation, spring-fifth grade: School year 2003-04 



Income category 


Before imputation 


Imputed 




After imputation 


Count 


Percent 


Count Percent 


Count 


Percent 


Total 


10,895 


100.0 


883 


100.0 


10,895 


100.0 


$5,000 or less 


196 


1.8 


53 


6.0 


249 


2.3 


$5,001-$10,000 


265 


2.4 


59 


6.7 


324 


3.0 


$10,001-$15,000 


484 


4.4 


87 


9.9 


571 


5.2 


$15,001-$20,000 


540 


5.0 


74 


8.4 


614 


5.6 


$20,001-$25,000 


602 


5.5 


87 


9.9 


689 


6.3 


$25,001-$30,000 


700 


6.4 


62 


7.0 


762 


7.0 


$30,001-$35,000 


575 


5.3 


49 


5.6 


624 


5.7 


$35,001-$40,000 


675 


6.2 


60 


6.8 


735 


6.8 


$40,001-$50,000 


966 


8.9 


91 


10.3 


1,057 


9.7 


$50,001-$75,000 


1,863 


17.1 


109 


12.3 


1,972 


18.1 


$75,001-$100,000 


1,448 


13.3 


80 


9.1 


1,528 


14.0 


$100,001-$200,000 


1,297 


11.9 


59 


6.7 


1,356 


12.5 


More than $200,000 


401 


3.7 


13 


1.5 


414 


3.8 


Missing 


883 


8.1 


t 


t 


0 


0.0 



f Not applicable. 

NOTE: Percentages may not sum to 100 due to rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten 
Class of 1998-99 (ECLS-K), spring 2004. 



The values of each SES component were then normalized so that the component has a mean 
of 0 and a standard deviation of 1. This is also known as the z-score. For the h- th SES component, a z- 
score zm for the /- th household was computed as 



where 



x hi 


is 


Wi 


is 


x w 


is 


( X J 


is 



the value of the h- th SES component for the /-th household; 
the base weight for the /-th record; 

the weighted mean of x hi ; and 

the standard error of x w . 



Note that where h is household income, x hi is the natural log of the midpoint of the detailed 
income range. The log of the detailed income range midpoint does not vary widely within the levels of the 
detailed income range, so the midpoint was a reasonable choice. It was used only for the purpose of 
computing the SES composite and was not retained in the data file. 
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Thus, each component was converted to a z-score with mean of 0 and a standard deviation of 
one. The SES variable for the i - th household was computed as 

tflj 

I z u 

SES i =— , 

m i 

where 777 , is the number of nonmissing SES components for the i - th household. 

Table 5-20 shows the distribution of the SES values. As described, the SES composite is the 
average of up to five measures, each of which was standardized to have a mean of 0 and a standard 
deviation of 1, hence the negative values. Note that for households with only one parent present, not all 
the components were defined. In these cases, the SES was computed averaging the available components. 
In the fifth-grade data file, the continuous SES variable is W5SESL. 

A categorical SES variable (W5SESQ5) was created that contains the quintile for the value 
of the composite SES for the child. Quintile 1 represents the lowest SES category and quintile 5 
represents the highest SES category. The quintiles were computed at the child level using the fifth-grade 
child-level parent weights. Unweighted frequencies for this variable are given in table 5-21. 
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Table 5-20. Distribution of socioeconomic status (SES) values, 
spring-fifth grade: School year 2003-04 



Characteristic 


SES 


N 


10,895 


Mean 


-0.01 


Standard deviation 


0.81 


Minimum 


-2.48 


Maximum 


2.54 


Range 


5.02 


1 st percentile 


-1.87 


5 th percentile 


-1.29 


10 th percentile 


-1.04 


25 th percentile 


-0.68 


50 th percentile 


-0.21 


75 th percentile 


0.42 


90 th percentile 


0.98 


95 th percentile 


1.30 


99 th percentile 


1.90 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early 
Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 



Table 5-21. 



Unweighted frequencies of the socioeconomic status 
(SES) variable, spring-fifth grade: School year 



2003-04 



SES quintile 


Frequency 


Percent 


Total 


10,895 


100.0 


1 st quintile 


1,783 


16.4 


2 nd quintile 


2,000 


18.4 


3 rd quintile 


2,083 


19.1 


4 th quintile 


2,411 


22.1 


5 th quintile 


2,618 


24.0 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early 
Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 
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5.6 



Imputation of the School Lunch Composites 



The school lunch composites were computed at the school level for the set of public schools 
that have at least one child or parent respondent (i.e., the child has nonzero child weight, C6CW0, or 
nonzero child-level parent weight, C6PW0) in spring-fifth grade. There are two school lunch composites 
as follows: (1) Percent of children eligible for free school lunch; and, (2) Percent of children eligible for 
reduced-price lunch. For a description of how the data were collected and how the composites were 
computed, see section 7. 5. 4. 6 of the Combined User’s Manual for the ECLS-K Fifth-Grade Data Files 
and Electronic Codebooks (NCES 2006-032) (Tourangeau et al. forthcoming). 



Not all schools completed the school administrator questionnaire, and among those who did, 
not all responded to all three questions needed to compute the school lunch composites. Therefore, there 
were missing values for some of the components of the school lunch composite variables. Prior to fifth 
grade, if the source variables have missing value, then the composites were filled in with values computed 
using the most recent Common Core of Data (CCD) where available, or left missing. In fifth grade, the 
composites were computed as in the past, but if they had missing values, they were imputed. The source 
variables, however, were not imputed. Table 5-22 shows the level of missing data for the school lunch 
composite variables among the 2,008 public schools that have 9,328 child or parent respondents in the 
fifth grade of the ECLS-K. 



Table 5-22. Public schools and child-parent respondents with missing value of the school lunch 
composites, spring-fifth grade: School year 2003-04 





Number 


missing 


Percent missing 


School lunch composite 


Schools 


Child-parent 

respondents 


Schools 


Child-parent 

respondents 


Free lunch 


691 


2,545 


34.4 


27.3 


Reduced-price lunch 


712 


2,601 


35.5 


27.9 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class 
of 1998-99 (ECLS-K), spring 2004. 



Similar to the components for the SES composite, a two-stage procedure was used to impute 
missing values for each school lunch composite variable. First, if a school had nonmissing value of the 
school lunch composite in the kindergarten, first-grade, or third-grade year, missing values for the spring- 
fifth grade school lunch composites were filled in with values from the previous years. The rationale for 
this approach was that the best source of data for a school was the data from a previous year. 
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Second, data still missing after this initial step were imputed using a hot deck methodology. 
In hot deck imputation, the values computed for a school for both composites were assigned or “donated” 
to a “similar” school that had missing values of the composites. Imputation cells were created using the 
Title I status of the school (hard boundary) and the school latitude and longitude (soft boundaries). The 
imputed value for a school with a missing value was taken from the nearest neighbor defined by latitude 
and longitude within the hard-boundary cell (Title I status). The Title I status is a derived variable using 
the data on whether the school received Title I funds (S6TT1) and whether Title I funds were targeted or 
used school-wide (S6TT1TA), both collected in the school administrator questionnaire. If these two 
variables have missing values for fifth grade, then the most recent available data (from third grade or first 
grade or kindergarten) were used. If these data were missing from the school administrator questionnaire 
for all rounds, then the information from the most recent CCD (2002-03) was used. 

The resolution of cases having missing data is shown for each school lunch composite in 
table 5-23 (for schools) and table 5-24 (for child-parent respondents). Schools that were imputed by hot 
deck are generally transfer schools with few sample children in those schools. This is reflected in tables 
5-23 and 5-24 where the percent of children with hot-deck values of the school composites is much 
smaller than the percent of schools with hot-deck values of the school composites. 

Since children were designated as eligible for either free lunch or reduced-price lunch but 
not for both services, the two school lunch composites should sum to no more than 100 percent. A very 
small number of schools (less than 2 percent) had imputed values of the two school lunch composites 
summing to more than 100 percent. These values came from two sources: (1) from values reported by the 
school in another year, or (2) from the hot-deck imputation. The reporting error has been present in all 
rounds of the ECLS-K, and the decision was to keep the reported values in the data file. If the erroneous 
values came from the hot-deck imputation, then they were corrected so that the two school lunch 
composites do not add to more than 100 percent. Correction was made by capping the hot-decked values 
of the two lunch composite variables. This was done by multiplying each value by 100 divided by the 
sum of the two variables before capping. This way, both values were reduced by the same amount so they 
sum to 100 percent. 

Tables 5-25 to 5-28 show the characteristics of the school lunch composites before and after 
imputation, at the school level and at the child level. 
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Table 5-23. Imputation of school lunch composites at the school level, spring-fifth grade: School year 
2003-04 



School lunch composite 


Number 


Values from 
previous round 


Imputed by 
Hot deck 


missing 


n 


Percent 


n 


Percent 


Free lunch 


691 


256 


37.0 


435 


63.0 


Reduced-price lunch 


712 


265 


37.2 


447 


63.8 


SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class 


of 1998-99 (ECLS-K), spring 2004. 













Table 5-24. Imputation of school lunch composites at the child level, spring-fifth grade: School year 
2003-04 




Number 


Values from 
previous round 


Imputed by 
Hot deck 


School lunch composite 


missing 


n Percent 


n Percent 


Free lunch 
Reduced-price lunch 


2,545 

2,601 


1,777 69.8 

1,819 69.9 


768 30.2 

782 30.1 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class 
of 1998-99 (ECLS-K), spring 2004. 
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Table 5-25. Free lunch composite at the school level, before and after imputation, spring-fifth grade: 
School year 2003-04 



Characteristic 


Before imputation 


Imputed 


After imputation 


All cases 


2,008 


691 


2,008 


Number of missing 


691 


0 


0 


Mean (not including missing) 


38.75 


41.22 


39.60 


Standard deviation 


28.29 


29.79 


28.83 


Minimum 


0.00 


0.00 


0.00 


Maximum 


95.00 


95.00 


95.00 


Range 


95.00 


95.00 


95.00 


1 st percentile 


0.00 


0.00 


0.00 


5 th percentile 


2.33 


0.00 


2.06 


10 th percentile 


5.43 


1.52 


4.82 


25 th percentile 


14.10 


3.47 


14.12 


50 th percentile 


32.66 


14.20 


33.84 


75 th percentile 


60.41 


36.94 


62.52 


90 th percentile 


83.39 


85.71 


84.12 


95 th percentile 


95.00 


95.00 


93.75 


99 th percentile 


95.00 


95.00 


95.00 



NOTE: These statistics are from the restricted-use data file where some data have been masked according to the disclosure risk avoidance 
procedures. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class 
of 1998-99 (ECLS-K), spring 2004. 
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Table 5-26. Free lunch composite at the child level, before and after imputation, spring-fifth grade: 
School year 2003-04 



Characteristic 


Before imputation 


Imputed 


After imputation 


All cases 


9,328 


2,545 


9,328 


Number of missing 


2,545 


0 


0 


Mean (not including missing) 


37.12 


44.54 


39.15 


Standard deviation 


26.74 


32.13 


28.50 


Minimum 


0.00 


0.00 


0.00 


Maximum 


95.00 


95.00 


95.00 


Range 


95.00 


95.00 


95.00 


1st percentile 


0.00 


0.00 


0.00 


5th percentile 


2.17 


1.95 


2.08 


10th percentile 


5.48 


2.94 


5.00 


25th percentile 


15.43 


15.26 


15.26 


50th percentile 


31.13 


39.35 


33.25 


75th percentile 


53.57 


74.07 


60.41 


90th percentile 


80.62 


94.06 


84.98 


95th percentile 


90.38 


95.00 


94.42 


99th percentile 


95.00 


95.00 


95.00 



NOTE: These statistics are from the restricted-use data file where some data have been masked according to the disclosure risk avoidance 
procedures. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class 
of 1998-99 (ECLS-K), spring 2004. 
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Table 5-27. Reduced-price lunch composite at the school level, before and after imputation, spring- 
fifth grade: School year 2003-04 



Characteristic 


Before imputation 


Imputed 


After imputation 


All cases 


2,008 


712 


2,008 


Number of missing 


712 


0 


0 


Mean (not including missing) 


9.68 


9.37 


9.57 


Standard deviation 


10.75 


9.12 


10.36 


Minimum 


0.00 


0.00 


0.00 


Maximum 


100.00 


100.00 


100.00 


Range 


100.00 


100.00 


100.00 


1st percentile 


0.00 


0.00 


0.00 


5th percentile 


0.35 


0.00 


0.00 


10th percentile 


1.52 


0.67 


1.09 


25th percentile 


4.69 


4.05 


4.49 


50th percentile 


8.19 


8.24 


8.22 


75th percentile 


12.09 


12.00 


12.03 


90th percentile 


16.50 


16.67 


16.59 


95th percentile 


20.00 


20.26 


20.12 


99th percentile 


91.37 


44.78 


50.00 



NOTE: These statistics are from the restricted-use data file where some data have been masked according to the disclosure risk avoidance 
procedures. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class 
of 1998-99 (ECLS-K), spring 2004. 
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Table 5-28. Reduced-price lunch composite at the child level, before and after imputation, spring-fifth 
grade: School year 2003-04 



Characteristic 


Before imputation 


Imputed 


After imputation 


All cases 


9,328 


2,601 


9,328 


Number of missing 


2,601 


0 


0 


Mean (not including missing) 


9.06 


9.05 


9.06 


Standard deviation 


8.13 


11.38 


9.15 


Minimum 


0.00 


0.00 


0.00 


Maximum 


100.00 


100.00 


100.00 


Range 


100.00 


100.00 


100.00 


1st percentile 


0.00 


0.00 


0.00 


5th percentile 


0.26 


0.00 


0.00 


10th percentile 


1.53 


0.00 


0.96 


25th percentile 


4.85 


3.02 


4.36 


50th percentile 


8.16 


7.71 


8.00 


75th percentile 


11.84 


11.72 


11.80 


90th percentile 


15.53 


16.07 


15.60 


95th percentile 


18.36 


19.53 


18.53 


99th percentile 


31.25 


73.68 


39.57 



NOTE: These statistics are from the restricted-use data file where some data have been masked according to the disclosure risk avoidance 
procedures. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, 
Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 
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6. RESPONSE RATES 



This chapter describes the computation of unit completion rates for the spring-fifth grade 
data collection of the ECLS-K, and unit overall response rates for the base year respondents. Weighted 
and unweighted unit completion rates are presented for three groups of children: (1) children sampled in 
kindergarten, (2) children sampled in first grade through the freshening procedure, and (3) both groups 
combined. Completion rates for the fifth-grade data collection were computed with the same procedures 
used for spring-first grade and spring-third grade to allow for comparisons of completion rates for the 
three years of data collection following the base year. Item response rates for selected items from the 
ECLS-K fifth-grade instruments are also presented. 

For spring-first grade and spring-third grade, the sample of children is the same: base-year 
respondents (i.e., children who had either a fall- or spring-kindergarten child assessment or parent 
interview) and children sampled in spring-first grade as part of sample freshening as described in section 
3.4.2. For spring-fifth grade, the sample of children was reduced to exclude base-year respondents who 
belonged in the following special groups as described in section 3.6: (1) children who became ineligible 
in an earlier round (because they died or moved out of the country), (2) children who were subsampled 
out in previous rounds because they moved out of the original schools and were not subsampled to be 
followed, (3) children whose parents emphatically refused to cooperate (hard refusals) in any of the data 
collection rounds since spring-kindergarten, and (4) children eligible for the third-grade sample for whom 
there are neither first-grade nor third-grade data. Among the 21,357 children who were eligible for the 
study after the base year, 16,143 were part of the fifth-grade data collection. 



6.1 Definition of Response and Completion Rates 

Response rates and completion rates are two ways to describe the outcomes of data 
collection activities. A response rate is the ratio of the number of units with completed interviews (for 
example, the units could be children, parents, schools or teachers) to the number of units sampled and 
eligible for the interview. The response rate indicates the percentage of possible interviews completed, 
taking all survey stages into account. On the other hand, the completion rate measures the percentage of 
interviews completed for a specific stage of the survey. For example, in the base year of the ECLS-K 
children were identified for assessment in a two-stage process. The first stage involved the recruitment of 
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schools to participate in the study. Preassessment visits were made to schools that agreed to participate. 
During the preassessment visit, field supervisors met with the participating school’s school coordinator to 
enumerate and sample the kindergartners. Assessments were then conducted for the sampled children whose 
parents consented. If the school refused to participate in the study, no children were sampled for assessment. 
Under this design, the completion rate for the child assessment is the percentage of sampled children who 
completed the assessment. The response rate is the product of the school participation or cooperation rate 
and the child assessment completion rate. 

Response and completion rates can be either unweighted or weighted. The unweighted rate, 
computed using the raw number of cases, provides a useful description of the success of the operational 
aspects of the survey. The weighted rate, computed by summing the weights (usually the reciprocals of 
the probability of selecting the units) for both the numerator and denominator, gives a better description 
of the success of the survey with respect to the population sampled since the weights allow for inference 
of the sample data (including response status) to the population. Both rates are usually not very different 
unless the probabilities of selection and the response rates in the categories with different selection 
probabilities vary considerably. 

For example, the weighted completion rate of the ECLS-K child assessment (CA) is 

computed as 



r CA 



Z w t 

i^ER CA 

Z w i 



i eER CA vjENR ca 



where W t is the weight (inverse of the probability of selection of the child) for child i, and ER C a denotes 
eligible child assessment respondent and ENRca eligible child assessment nonrespondent. To compute the 
unweighted rates, W, is set to 1 for each child. 

The response rate of the child assessment can be computed as 



R CA - r S x ' CA 



where r s is the school cooperation rate and r C A is the child assessment completion rate. 
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After the base year, only completion rates were computed for the different ECLS-K 
instruments, since the response rates of the schools where the children were sampled remained the same 
for each subsequent round. Data users can compute the fifth-grade response rate for each ECLS-K 
instrument by multiplying the school response rate from the base year and the fifth-grade completion rate 
for each instrument. 

Both unweighted and weighted rates are presented in the tables of completion rates in this 
chapter. While unweighted rates are useful for evaluating sample performance as mentioned earlier, only 
weighted rates are discussed in the text. 



6.2 Completion Rates 

For the ECLS-K fifth-grade data collection, there were 1 1 survey instruments: child 
assessment; parent interview; school administrator questionnaire; facilities checklist; student records 
abstract, teacher-level questionnaire, subject-specific child-level questionnaires (reading, mathematics, 
and science); and special education teacher questionnaire part A and part B. The mathematics teacher 
questionnaire was completed for about half of the children in the fifth-grade sample and the science 
teacher questionnaire was completed for the other half, so that each child would have data from no more 
than 10 instruments. Except for the child assessment and the parent interview, all other instruments were 
paper-and-pencil instruments. 

For each instrument, completion rates were computed separately for children who were 
sampled as part of the kindergarten cohort in the base year and for children who were sampled in first 
grade through the student sample freshening procedure. While the completion rate for children sampled in 
the base year has only one component (to account for nonresponse attrition during data collection), the 
completion rate for children sampled in first grade has two components (to account for nonresponse 
attrition during the freshening procedure and during data collection). Section 6.2.2 describes in detail the 
two components of the completion rates for the freshened children. The two sets of rates were combined 
to obtain the completion rates for all children in fifth grade. Movers who were sampled to be followed but 
could not be located are considered nonrespondents and included in the denominator of the computation 
of completion rates. 
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6.2.1 



Children Sampled in Kindergarten 



For the ECLS-K, a completion rate is a response rate conditioned on the results of an earlier 
stage of data collection. In the case of the ECLS-K fifth-grade data collection, the condition is that 
children who were sampled in kindergarten were base year respondents since only base year respondents 
were eligible for subsequent data collection efforts. Children sampled in first grade were exempt from this 
condition in the computation of completion rates. They are discussed in section 6.2.2. 

For each instrument, the unweighted completion rate is the proportion of base year 
respondents with completed data for the instrument to the base-year respondents who remain eligible to 
have the fifth-grade instrument administered. Base-year respondents who were subsampled out because 
they moved from their base- year original sample schools and base-year respondents who died or moved 
out of the country were not included in the denominator. For the weighted completion rates, the weight 
used is the product of the school base weight, the within-school child weight, and the factor that was used 
to adjust for movers between base year and fifth grade who were subsampled out for data collection. For a 
description of these weights and adjustment factor, see chapter 7. 

Tables 6-1 to 6-4 present weighted and unweighted child-level completion rates for spring- 
fifth-grade data collection, broken out by school characteristics. 1 These rates pertain to children who were 
sampled as part of the kindergarten cohort in the base year. In general, completion rates for fifth grade are 
higher than they had been for the third grade. This is due to the exclusion of hard-to-field cases from the 
fifth-grade collection. Hard-to-field cases are the hard-refusal cases and cases that were nonrespondents in 
both first and third grades as described in section 3.6. If these cases had not been excluded from the fifth 
grade, they would most likely be nonrespondents and would bring down the completion rates. 

Table 6-1 shows that the completion rates for the child assessment are quite high and 
uniform across school characteristics. Excluding the “u nk nown” category, the rates range from 93.1 
percent in non-Catholic private schools to 99.7 percent in schools in large town. Similarly, the completion 
rates for the parent interviews were uniform across school characteristics ranging from 87.2 percent for 
children in schools with total enrollment of 750 or more, and in schools where 50 to 89 percent of 



1 Children in schools with unknown characteristics are movers who could not be located (and considered as nonrespondents in the completion 
rates). Their weights are large because of the mover adjustment where movers who were followed carry the weight of movers who were 
subsampled-out for follow-up. The categories of school affiliation in the tables in this chapter do not match categories of school affiliation in the 
tables in chapter 3. This is to allow users to compare completion rates in fifth grade with those in previous years. 
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Table 6-1. Number of completed child-level cases and child- level completion rates for the child assessment and parent interview for children 
sampled in the base year, by school characteristics: School year 2003-04 







Child assessment 






Parent interview 




School characteristics 1 




Completion rates 




Completion rates 




Completes 1 


Weighted 


Unweighted 


Completes 3 


Weighted Unweighted 


All schools 


11,260 


84.7 


93.6 


10,913 


89.1 


90.7 


School affiliation 


Public 


9,187 


96.2 


97.7 


8,518 


89.9 


90.6 


Private 


2,049 


95.2 


97.7 


1,977 


92.0 


94.3 


Catholic 


1,313 


97.1 


98.3 


1,260 


94.0 


94.3 


Other private 


736 


93.1 


96.7 


717 


89.5 


94.2 


Unknown 


24 


5.8 


4.6 


418 


82.5 


79.5 


Type of locale 


Large city 


1,863 


97.2 


97.6 


1,697 


89.6 


88.9 


Mid-size city 


1,863 


97.6 


98.3 


1,733 


90.8 


91.5 


Urban fringe of large city 


3,286 


94.8 


96.6 


3,085 


88.6 


90.7 


Urban fringe of mid-size city 


767 


93.5 


96.8 


725 


90.3 


91.5 


Large town 


283 


99.7 


99.6 


270 


91.9 


95.1 


Small town 


825 


94.1 


98.8 


781 


94.3 


93.5 


Rural - outside MSA 


1,286 


97.2 


97.9 


1,205 


88.1 


91.7 


Rural - inside MSA 


922 


97.7 


99.4 


871 


93.4 


93.9 


Unknown 


165 


17.0 


24.6 


546 


83.4 


81.5 


School size (total enrollment) 


1 to 299 


2,359 


96.7 


98.2 


2,255 


93.4 


93.8 


300 to 499 


3,703 


96.5 


98.0 


3,453 


89.9 


91.4 


500 to 749 


3,167 


97.1 


97.6 


2,945 


90.8 


90.8 


750 or more 


1,963 


94.2 


97.0 


1,798 


87.2 


88.9 


Unknown 


68 


9.7 


11.7 


462 


82.1 


79.8 



See notes at end of table. 



Table 6-1. Number of completed child-level cases and child- level completion rates for the child assessment and parent interview for children 
sampled in the base year, by school characteristics: School year 2003-04 — Continued 







Child assessment 






Parent interview 








Completion rates 




Completion rates 




School characteristics 1 


Completes^ 


Weighted 


Unweighted 


Completes 3 


Weighted Unweighted 


Percent non- White enrolled 


0-10 


3,509 


97.9 


98.3 


3,352 


93.2 


93.9 


11-49 


3,705 


94.5 


97.2 


3,522 


90.5 


92.4 


50-89 


1,956 


96.5 


97.4 


1,768 


87.2 


88.0 


90-100 


1,997 


97.0 


98.0 


1,786 


88.2 


87.7 


Unknown 


93 


11.3 


15.4 


485 


82.4 


80.3 


Region 


Northeast 


2,080 


95.5 


96.8 


1,956 


90.5 


91.1 


Midwest 


2,957 


97.9 


98.8 


2,803 


93.0 


93.6 


South 


3,614 


95.1 


97.5 


3,334 


87.6 


90.0 


West 


2,585 


96.5 


97.3 


2,402 


91.2 


90.4 


Unknown 


24 


5.8 


4.6 


418 


82.5 


79.5 



1 School characteristics are for schools attended by children in the ECLS-K fifth-grade sample and are based on ECLS-K survey data, not data from the sampling frame. 

2 Reading, mathematics, or science assessments was scorable, or child was disabled and could not be assessed. 

3 Family structure portion of parent interview was completed. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 



children belong to the minority groups, to 94.3 percent for children in small towns (excluding the 
“u nk nown” category). The “unknown” category includes children who were unbeatable as their 
whereabouts were u nk nown and those children who had moved into a nonsampled county. The category 
“u nk nown” also includes 35 children who were homeschooled and thus had no information concerning 
schools. 



The “u nk nown” category aside, both the child assessment and the parent interview 
completion rates increased between third grade and fifth grade for all school characteristics. The 
completion rates by mover status are discussed later, but the rates of completing all the instruments are 
much lower for children who moved than for those who did not move. 

Table 6-2 shows that the overall weighted completion rate is 77.1 percent for the school 
administrator questionnaire, and 78.8 percent for the facilities checklist. The rate for school administrator 
questionnaires is 1 1 percentage points higher than the corresponding rate in third grade. The rate for 
facilities checklist is only about 2 percent higher. Excluding the “u nk nown” category, the completion 
rates for the school administrator questionnaire range from 87.4 percent for schools with 750 or more 
students to 100 percent for those in large towns. Rates for the facilities checklist range from 90.3 percent 
for schools in the urban fringe of mid-size cities to 100 percent for schools in large towns. It is worth 
noting that the completion rates for the school administrator questionnaire are lower for schools with 
higher percentages of minorities, a phenomenon also observed in previous rounds for the school 
administrator questionnaire. However, this disparity decreased considerably after the base year, reflecting 
the success of increased data collection efforts targeted toward these schools. 
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Table 6-2. Number of completed child-level cases and child-level completion rates for the school administrator questionnaire and facilities 
checklist for children sampled in the base year, by school characteristics: School year 2003-04 





School administrator questionnaire 






Facilities checklist 








Completion rates 






Completion rates 




School characteristics 1 


Completes 9 


Weighted Unweighted 


Completes 2 


Weighted Unweighted 


All schools 


10,937 


77.1 


89.6 


11,154 


78.8 


91.4 


School affiliation 


Public 


8,884 


90.7 


94.5 


9,084 


92.7 


96.6 


Private 


2,053 


93.1 


97.9 


2,070 


95.7 


98.7 


Catholic 


1,323 


96.8 


99.0 


1,328 


98.1 


99.4 


Other private 


730 


88.8 


95.9 


742 


92.8 


97.5 


Type of locale 


Large city 


1,775 


92.2 


93.0 


1,862 


96.4 


97.6 


Mid-size city 


1,825 


93.6 


96.3 


1,844 


94.9 


97.3 


Urban fringe of large city 


3,173 


87.7 


93.2 


3,240 


90.4 


95.2 


Urban fringe of mid-size city 


762 


90.9 


96.2 


761 


90.3 


96.1 


Large town 


284 


100.0 


100.0 


284 


100.0 


100.0 


Small town 


823 


91.2 


98.6 


823 


91.1 


98.6 


Rural - outside MSA 


1,274 


92.0 


97.0 


1,292 


93.6 


98.3 


Rural - inside MSA 


904 


93.9 


97.4 


911 


94.6 


98.2 


Unknown 


117 


8.2 


13.9 


137 


9.4 


16.3 


School size (total enrollment) 


1 to 299 


2,360 


95.1 


98.2 


2,368 


95.5 


98.5 


300 to 499 


3,662 


92.0 


96.9 


3,684 


92.6 


97.5 


500 to 749 


3,071 


92.3 


94.6 


3,136 


94.3 


96.6 


750 or more 


1,844 


87.4 


91.2 


1,939 


91.9 


95.8 


Unknown 


0 


0.0 


0.0 


27 


1.7 


3.6 



See notes at end of table. 



Table 6-2. Number of completed child-level cases and child-level completion rates for the school administrator questionnaire and facilities 
checklist for children sampled in the base year, by school characteristics: School year 2003-04 — Continued 



School administrator questionnaire Facilities checklist 



Completion rates Completion rates 



School characteristics 1 


Completes^ 


Weighted 


Unweighted 


Completes 1 


Weighted 


Unweighted 


Percent non- White enrolled 


0-10 


3,502 


95.0 


98.1 


3,515 


95.5 


98.5 


11-49 


3,633 


89.8 


95.4 


3,674 


91.1 


96.4 


50-89 


1,864 


90.4 


92.8 


1,920 


92.9 


95.6 


90-100 


1,916 


92.4 


94.1 


1,996 


96.2 


98.0 


Unknown 


22 


0.9 


2.8 


49 


2.5 


6.3 


Region 


Northeast 


2,008 


87.7 


93.5 


2,048 


90.4 


95.3 


Midwest 


2,920 


95.5 


97.5 


2,938 


95.9 


98.1 


South 


3,547 


90.7 


95.7 


3,598 


92.1 


97.1 


West 


2,462 


89.1 


92.7 


2,570 


93.9 


96.8 



1 School characteristics are for schools attended by children in the ECLS-K third-grade sample and are based on ECLS-K survey data, not data from the sampling frame. 

2 A completed questionnaire was defined as one that was not left completely blank. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 



Table 6-3 shows that the rates for the student records abstract are the lowest of all the 
instruments, as they always were in previous years of the ECLS-K. For fifth grade, this rate is about 70 
percent compared with 67 percent in third grade. The “u nk nown” category aside, the completion rates of 
the student records abstract range from 71.8 percent in the northeast region to 93.7 percent for children in 
large towns. 



All four of the teacher questionnaires were completed at an overall rate of 78 to 80 percent 
(tables 6-3 and 6-4), much higher than the 62 to 63 percent range achieved in third grade. Excluding the 
“u nk nown” category, the completion rates for the teacher-level questionnaire in table 6-3 are uniform 
across school characteristics, ranging from 86.9 percent for schools in the northeast and schools with 750 
or more students to 99.7 percent for schools in large towns. The same uniform rates are found for the 
subject-specific child-level teacher questionnaires in table 6-4: 85.7 to 99.7 percent for reading, 81.7 to 
99.4 percent for mathematics, and 86.8 to 100 percent for science. These rates are higher than in any 
previous years of the ECLS-K, a change most likely attributable to the higher incentives employed in fifth 
grade. For a discussion of the incentives used in the ECLS-K, see section 4.5.6. 

As noted above, the rate at which the survey instruments were completed varies markedly by 
mover status and, within movers, by whether the child was located and followed. As shown in table 6-5, 
the completion rate for the child assessment was 98.2 percent for children still enrolled in their base year 
school. For movers it dropped by about 6 points to 91.9 percent for those who were located and followed, 
and for those not located or followed due to a move to a non-ECLS-K PSU, it was zero. The parent 
interview completion rates varied from 91.6 percent for nonmovers to 87.1 percent for movers who were 
located and followed for the purposes of the child assessment, to 85.7 percent for movers who could 
either not be located or were not followed for the purposes of the child assessment. Even though children 
who had moved to a non-ECLS-K PSU were not administered the child assessment, a parent interview 
was conducted by telephone wherever possible, leading to the 86 percent response rate for this category. 
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Table 6-3. Number of completed child-level cases and child- level completion rates for the student records abstract and teacher-level 
questionnaire for children sampled in the base year, by school characteristics: School year 2003-04 





Student records abstract 




Teacher-level questionnaire 




School characteristics 1 




Completion rates 






Completion rates 




Completes 1 


Weighted 




Completes 1 


Weighted Unweighted 


All schools 


10,015 


69.9 


82.1 


10,872 


79.9 


90.6 


School affiliation 


Public 


8,177 


82.3 


86.9 


8,849 


90.2 


94.1 


Private 


1,838 


84.6 


87.6 


2,023 


93.6 


96.5 


Catholic 


1,209 


90.3 


90.5 


1,313 


97.0 


98.3 


Other private 


629 


77.9 


82.7 


710 


89.4 


93.3 


Type of locale 


Large city 


1,585 


78.9 


83.1 


1,758 


89.8 


92.1 


Mid-size city 


1,688 


85.3 


89.1 


1,819 


94.1 


96.0 


Urban fringe of large city 


2,778 


76.8 


81.6 


3,131 


87.4 


92.0 


Urban fringe of mid-size city 


698 


82.1 


88.1 


751 


90.2 


94.8 


Large town 


272 


93.7 


95.8 


283 


99.7 


99.6 


Small town 


758 


85.7 


90.8 


819 


91.1 


98.1 


Rural - outside MSA 


1,227 


89.3 


93.4 


1,273 


91.0 


96.9 


Rural - inside MSA 


884 


89.4 


95.3 


903 


93.4 


97.3 


Unknown 


125 


8.8 


14.9 


135 


12.0 


21.3 


School size (total enrollment) 


1 to 299 


2,114 


85.2 


88.0 


2,340 


94.8 


97.4 


300 to 499 


3,385 


84.1 


89.6 


3,628 


91.3 


96.0 


500 to 749 


2,808 


83.3 


86.5 


3,039 


91.4 


93.7 


750 or more 


1,687 


78.6 


83.4 


1,844 


86.9 


91.2 


Unknown 


21 


1.3 


2.8 


21 


1.7 


3.9 



See note at end of table. 
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Table 6-3. Number of completed child-level cases and child- level completion rates for the student records abstract and teacher-level 
questionnaire for children sampled in the base year, by school characteristics: School year 2003-04 — Continued 





Student records abstract 




Teacher-level questionnaire 








Completion rates 






Completion rates 




School characteristics 1 


Completes^ 


Weighted Unweighted 


Completes 3 


Weighted Unweighted 


Percent non- White enrolled 


0-10 


3,274 


89.0 


91.7 


3,465 


93.9 


97.1 


11-49 


3,303 


81.4 


86.7 


3,608 


89.2 


94.7 


50-89 


1,686 


80.2 


84.0 


1,854 


90.5 


92.3 


90-100 


1,712 


80.2 


84.0 


1,902 


91.4 


93.4 


Unknown 


40 


2.1 


5.2 


43 


2.8 


7.6 


Region 


Northeast 


1,635 


71.8 


76.1 


1,970 


86.9 


91.7 


Midwest 


2,788 


90.4 


93.1 


2,913 


94.7 


97.3 


South 


3,359 


83.4 


90.7 


3,531 


90.1 


95.3 


West 


2,233 


81.3 


84.1 


2,458 


90.0 


92.5 



1 School characteristics are for schools attended by children in the ECLS-K third-grade sample and are based on ECLS-K survey data, not data from the sampling frame. 

2 Reading, mathematics, or science assessments was scorable, or child was disabled and could not be assessed. 

3 Family structure portion of parent interview was completed. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 
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Table 6-4. Number of completed child-level cases and child-level completion rates for the child-level teacher questionnaires for children sampled in 
the base year, by school characteristics: School year 2003-04 





Child-level 

reading teacher questionnaire 




Child-level 

mathematics teacher questionnaire 


science 


Child-level 
teacher questionnaire 






Completion rates 






Completion rates 




Completion rates 


School characteristics 1 


Completes 2 


Weighted Unweighted 


Completes 2 


Weighted 


Unweighted 


Completes 2 


Weighted 


Unweighted 


All schools 


10,793 


79.3 


90.0 


5,339 


78.1 


89.3 


5,405 


79.5 


89.9 


School affiliation 


Public 


8,780 


89.5 


93.3 


4,357 


87.9 


92.6 


4,376 


89.8 


93.1 


Private 


2,013 


92.9 


96.0 


982 


93.6 


96.0 


1,029 


92.1 


95.8 


Catholic 


1,306 


96.7 


97.8 


635 


96.5 


97.2 


670 


96.7 


98.1 


Other private 


707 


88.5 


92.9 


347 


90.2 


93.8 


359 


86.6 


91.8 


Type of locale 


Large city 


1,753 


90.2 


91.9 


853 


90.3 


91.8 


880 


88.0 


89.9 


Mid-size city 


1,811 


93.6 


95.6 


893 


90.3 


94.3 


898 


94.0 


94.7 


Urban fringe of large city 


3,111 


86.5 


91.4 


1,547 


84.4 


90.4 


1,551 


87.9 


91.7 


Urban fringe of mid-size city 


741 


88.3 


93.6 


366 


87.6 


93.1 


374 


88.0 


93.7 


Large town 


283 


99.7 


99.6 


140 


99.4 


99.3 


143 


100.0 


100.0 


Small town 


808 


90.3 


96.8 


383 


85.0 


96.5 


432 


96.8 


98.6 


Rural - outside MSA 


1,258 


90.1 


95.7 


634 


92.8 


95.5 


620 


86.8 


95.4 


Rural - inside MSA 


893 


91.8 


96.2 


456 


90.5 


95.6 


441 


94.3 


97.8 


Unknown 


135 


12.0 


21.3 


67 


13.1 


20.9 


66 


10.5 


21.0 


School size (total enrollment) 


1 to 299 


2,323 


93.8 


96.7 


1,144 


94.2 


96.6 


1,175 


93.7 


96.4 


300 to 499 


3,605 


91.0 


95.4 


1,830 


90.7 


95.8 


1,768 


90.2 


94.6 


500 to 749 


3,014 


90.6 


92.9 


1,462 


88.8 


91.7 


1,537 


91.3 


93.2 


750 or more 


1,830 


85.7 


90.5 


892 


81.7 


88.0 


917 


88.0 


90.9 


Unknown 


21 


1.7 


3.9 


11 


2.2 


4.0 


8 


0.7 


3.0 



See notes at end of table. 
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Table 6-4. Number of completed child-level cases and child-level completion rates for the child-level teacher questionnaires for children sampled in 
the base year, by school characteristics: School year 2003-04 — Continued 



School characteristics 1 


Child-level 

reading teacher questionnaire 




Child-level 

mathematics teacher questionnaire 


science 


Child-level 
: teacher questionnaire 




Completion rates 




Completes 2 


Completion rates 




Completion rates 


Completes 2 


Weighted 


Unweighted 


Weighted Unweighted Completes 2 


Weighted 


Unweighted 


Percent non- White enrolled 






















0-10 


3,444 


93.7 




96.5 


1,695 


93.2 


96.3 


1,745 


94.1 


96.5 


11-49 


3,584 


88.1 




94.1 


1,791 


87.2 


93.8 


1,793 


89.0 


94.4 


50-89 


1,823 


89.1 




90.8 


919 


85.8 


89.7 


893 


90.8 


90.8 


90-100 


1,899 


91.3 




93.2 


911 


90.4 


91.7 


956 


88.8 


91.6 


Unknown 


43 


2.8 




7.6 


23 


3.3 


7.9 


18 


1.9 


6.5 


Region 






















Northeast 


1,956 


86.7 




91.1 


967 


84.6 


90.6 


969 


87.0 


89.6 


Midwest 


2,909 


94.4 




97.2 


1,423 


94.5 


96.9 


1,482 


94.5 


97.1 


South 


3,498 


89.0 




94.4 


1,747 


87.5 


93.6 


1,743 


89.2 


94.8 


West 


2,430 


89.0 




91.5 


1,202 


87.4 


90.5 


1,211 


89.1 


91.2 



1 School characteristics are for schools attended by children in the ECL-K third-grade sample and are based on ECLS-K survey data, not data from the sampling frame. 

2 A completed questionnaire was defined as one that was not left completely blank. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 



6-15 



Table 6-5. Number of completed child-level cases and child-level completion rates for the child assessment and parent 
interview for children sampled in the base year, by child’s mover status: School year 2003-04 



Mover status 1 


Child assessment 




Parent interview 


Completes 2 


Completion rates 


Completes 3 


Completion rates 


Weighted 


Unweighted 


Weighted 


Unweighted 


All children 


11,260 


84.7 


93.6 


10,913 


89.1 


90.7 


Mover status 














Mover 


1,814 


71.8 


74.7 


2,094 


86.8 


86.2 


Located, followed 


1,814 


91.9 


92.1 


1,704 


87.1 


86.5 


Other 4 


0 


0.0 


0.0 


390 


85.7 


85.0 


Nonmover 


9,446 


98.2 


98.4 


8,819 


91.6 


91.9 



1 This is the mover status used in weighting, which does not consider children who moved into identified destination schools as movers. A destination school is a 
school that received at least four students from the school where they had just completed the highest grade. 

2 Reading, mathematics, or science assessments was scorable, or child was disabled and could not be assessed. 

3 Family structure portion of parent interview was completed. 

4 This category includes movers who could not be located, and movers who moved into nonsampled PSUs. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), 
spring 2004. 



The school administrator questionnaire completion rate is 15 points lower for movers, even 
when the children were located and followed; for the facilities checklist, it is 14 points lower (table 6-6). 
There are several reasons for this difference: located movers were not always assessed in schools; new 
schools in which movers enrolled had a lower level of commitment to the ECLS-K and often refused to 
complete the school administrator questionnaire; and some of these schools were contacted too late in the 
school year for them to consider completing it. The completion rate for nonmovers was 97. 1 percent for 
the school administrator questionnaire and 98.8 percent for the facilities checklist. For located and 
followed movers it was 82.4 and 84.8 percent for the school administrator questionnaire and for the 
facilities checklist, respectively. The rates for the student records abstract are 90.1 percent for nonmovers 
and 72.3 percent for movers who were located and followed (table 6-7). 

The teacher-level questionnaire completion rate, as shown in table 6-7, is about 14 points 
lower for movers who could be located and followed (82.2 percent) than for nonmovers (96.5 percent). 
Movers who could not be located were all nonrespondents for this instrument, pulling the overall 
completion rate for movers downward to 63.9 percent. Table 6-8 shows the completion rates for all three 
child-level teacher questionnaires. These rates are between 95 and 96 percent for nonmovers, and between 
80 and 82 percent for movers who were located and followed. Children who could not be located were all 
nonsrespondents for the child-level teacher instruments. The reasons for lower completion rates from 
teachers if the child moved are similar to the reasons that affected the school administrator questionnaire 
and facilities checklist completion rates for movers. 
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Table 6-6. 



Number of completed child-level cases and child-level completion rates for the school administrator 
questionnaire and facilities checklist for children sampled in the base year, by child’s mover status: School year 



2003-04 





School administrator questionnaire 


Facilities checklist 


1 




Completion rates 




Completion rates 


Mover status 


Completes 2 


Weighted 


Unweighted 


Completes^ 


Weighted 


Unweighted 


All children 


10,937 


77.1 


89.6 


11,154 


78.8 


91.4 


Mover status 


Mover 


1,589 


59.2 


61.1 


1,647 


61.0 


63.3 


Located, followed 


1,589 


82.4 


82.1 


1,647 


84.8 


85.1 


Other 3 


0 


0.0 


0.0 


0 


0.0 


0.0 


Nonmover 


9,348 


97.1 


97.4 


9,507 


98.8 


99.0 



1 This is the mover status used in weighting, which does not consider children who moved into identified destination schools as movers. A destination school is a 
school that received at least four students from the school where they had just completed the highest grade. 

2 A completed questionnaire was defined as one that was not left completely blank. 

3 This category includes movers who could not be located, and movers who moved into nonsampled PSUs. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 
(ECLS-K), spring 2004. 
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Table 6-7. Number of completed child-level cases and child-level completion rates for the student records abstract and 

teacher-level questionnaire for children sampled in the base year, by child’s mover status: School year 2003-04 



Mover status 1 


Student records abstract 


Teacher-level questionnaire 




Completion rates 


Completes 1 


Completion rates 


Completes^ 


Weighted 


Unweighted 


Weighted 


Unweighted 


All children 


10,015 


69.9 


82.1 


10,872 


79.9 


90.6 


Mover status 














Mover 


1,393 


52.0 


53.6 


1,587 


63.9 


66.3 


Located, followed 


1,393 


72.3 


72.0 


1,587 


82.2 


82.0 


Other 3 


0 


0.0 


0.0 


0 


0.0 


0.0 


Nonmover 


8,622 


90.1 


89.8 


9,285 


96.5 


96.7 



1 This is the mover status used in weighting, which does not consider children who moved into identified destination schools as movers. A destination school 
is a school that received at least four students from the school where they had just completed the highest grade. 

2 A completed questionnaire was defined as one that was not left completely blank. 

3 This category includes movers who could not be located, and movers who moved into nonsampled PSUs. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), 
spring 2004. 
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Table 6-8. Number of completed child-level cases and child-level completion rates for the child-level teacher questionnaires for children sampled in the 
base year, by child’s mover status: School year 2003-04 



Mover status 1 


Child-level 

reading teacher questionnaire 


Child-level 

mathematics teacher questionnaire 


science 


Child-level 
teacher questionnaire 




Completion rates 




Completion rates 


Completes 2 


Completion rates 


Completes 2 


Weighted 


Unweighted 


Completes 2 


Weighted 


Unweighted 


Weighted 


Unweighted 


All children 


10,793 


79.3 


90.0 


5,339 


78.1 


89.3 


5,405 


79.5 


89.9 


Mover status 




















Mover 


1,568 


63.3 


65.5 


818 


62.2 


65.2 


734 


63.2 


64.4 


Located, followed 


1,568 


81.4 


81.0 


818 


79.7 


80.4 


734 


81.7 


80.0 


Other 3 


0 


0.0 


0.0 


0 


0.0 


0.0 


0 


0.0 


0.0 


Nonmover 


9,225 


95.8 


96.1 


4,521 


95.3 


95.7 


4,671 


95.6 


95.8 



1 This is the mover status used in weighting, which does not consider children who moved into identified destination schools as movers. A destination school is a school that received at least four 
students from the school where they had just completed the highest grade. 

2 A completed questionnaire was defined as one that was not completely left blank. 

3 This category includes movers who could not be located, and movers into nonsampled PSUs. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 



Tables 6-9 to 6-12 present child-level weighted and unweighted completion rates for the 
spring-fifth grade data collection for children who were sampled as part of the kindergarten cohort in the 
base year, this time broken out by child characteristics. When the “u nk nown” categories are not included, 
the differences in completion rates by sex and by year of birth are inconsequential, but for race and 
ethnicity they are more substantial. Table 6-9 shows that the child assessment completion rate was highest 
for Asians (87.6 percent) and lowest for American Indians or Alaska Natives (78.3 percent). For the 
parent interview it is the opposite; the rate was highest for American Indians or Alaska Natives (95.2 
percent) and lowest for Asian children (82.8 percent). 

Table 6-10 shows that, excluding the “u nk nown” categories, the highest completion rates for 
the school administrator questionnaire and for the facilities checklist are for Pacific Islanders (85.7 
percent and 86.7 percent, respectively), and the lowest completion rates are for American Indians or 
Alaska Natives (65.8 percent and 71.1 percent, respectively). Table 6-11 shows that the completion rate 
for the student records abstract is highest for children with “other” race (72.8 percent) and the lowest is 
for Black (63.9), excluding the “unknown” categories. 

For the teacher-level questionnaire (table 6-11), the highest rate is for Pacific Islanders (84 
percent) and the lowest rate is for American Indians and Alaska Natives (77 percent), excluding the 
“unknown” categories. For the child-level reading teacher questionnaire (table 6-12), the highest rate is 
for Asians (82 percent) and the lowest rate is for American Indians or Alaska Natives (76 percent). For 
the child-level mathematics teacher questionnaire, the highest rate is for Hispanic and for children whose 
race/ethnicity is not among the listed (80 percent) and the lowest rate is for Black and American Indians 
or Alaska Natives (76 percent). For the child-level science teacher questionnaire, the highest rate is for 
Asians (86 percent) and the lowest rate is for Blacks and for children whose race/ethnicity is not among 
the listed (76 percent). 

In addition to the child assessment, parent interview, school administrator questionnaire, 
facilities checklist, student records abstract, and teacher questionnaires, whose completion rates have been 
summarized in the preceding paragraphs, data were also collected in fifth grade from the special 
education teachers for children who had special education teachers. Table 6-13 presents counts of 
completes and weighted and unweighted completion rates at the overall student level for the special 
education questionnaires A and B. The number of special education teacher questionnaires is small but 
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Table 6-9. Number of completed child-level cases and child-level completion rates for the child assessment and parent interview for 
children sampled in the base year, by child characteristics: School year 2003-04 







Child assessment 






Parent interview 








Completion rates 




Completion rates 




Child characteristics 1 


Completes' 


Weighted 


Unweighted 


Completes 3 


Weighted Unweighted 


All children 


11,260 


84.7 


93.6 


10,913 


89.1 


90.7 


Sex 4 


Male 


5,675 


84.1 


93.3 


5,525 


89.9 


90.8 


Female 


5,585 


85.4 


94.1 


5,388 


88.5 


90.8 


Race/ethnicity 


White, non-Hispanic 


6,466 


84.8 


94.4 


6,394 


91.3 


93.4 


Black, non-Hispanic 


1,273 


83.3 


93.3 


1,151 


83.4 


84.3 


Hispanic 


2,093 


85.9 


92.4 


2,036 


89.5 


89.9 


Asian 


788 


87.6 


92.5 


707 


82.8 


83.0 


Pacific Islander 


144 


85.4 


92.3 


136 


87.7 


87.2 


American Indian or Alaska Native 


210 


78.3 


92.1 


222 


95.2 


97.4 


Other 


272 


86.3 


93.8 


253 


84.5 


87.2 


Unknown 


14 


55.3 


50.0 


14 


58.0 


50.0 


Y ear of birth 


1992 


3,307 


83.9 


93.6 


3,211 


88.7 


90.8 


1993 


7,896 


85.1 


93.8 


7,646 


89.4 


90.8 


Other/unknown 


57 


71.0 


76.0 


56 


76.0 


74.7 



1 Based on ECLS-K survey data and not on data from the sampling frame. 

2 Reading, mathematics, or science assessments was scorable, or child was disabled and could not be assessed. 

3 Family structure portion of parent interview was completed. 

4 There is a small number of children whose sex is unknown and who did not have completed child assessment and parent interview. The completion rates for these children, being zero, 
are not included in the table. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 
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Table 6-10. Number of completed child-level cases and child-level completion rates for the school administrator questionnaire and facilities 
checklist for children sampled in the base year, by child characteristics: School year 2003-04 





School administrator questionnaire 






Facilities checklist 








Completion rates 






Completion rates 




Child characteristics 1 


Completes 2 


Weighted Unweighted 


Completes 2 


Weighted Unweighted 


All children 


10,937 


77.1 


89.6 


11,154 


78.8 


91.4 


Sex 3 


Male 


5,517 


77.1 


89.5 


5,621 


78.6 


91.2 


Female 


5,420 


77.2 


90.0 


5,533 


79.2 


91.9 


Race/ethnicity 


White, non-Hispanic 


6,356 


78.8 


92.6 


6,413 


80.1 


93.4 


Black, non-Hispanic 


1,256 


74.5 


89.9 


1,266 


75.4 


90.6 


Hispanic 


1,957 


75.4 


83.5 


2,061 


79.0 


88.0 


Asian 


752 


77.9 


85.1 


776 


80.9 


87.8 


Pacific Islander 


145 


85.7 


92.4 


146 


86.7 


93.0 


American Indian or Alaska Native 


191 


65.8 


82.3 


208 


71.1 


89.7 


Other 


266 


77.7 


90.8 


270 


79.1 


92.2 


Unknown 


14 


50.1 


46.7 


14 


50.1 


46.7 


Y ear of birth 


1992 


3,223 


77.0 


90.0 


3,275 


78.7 


91.5 


1993 


7,660 


77.2 


89.7 


7,825 


79.1 


91.6 


Other/unknown 


54 


64.0 


70.1 


54 


59.5 


70.1 



1 Based on ECLS-K survey data and not on data from the sampling frame. 

2 A completed questionnaire was defined as one that was not left completely blank. 

3 The completion of the school-level instruments does not depend on whether the child has completed assessment or parent interview data. Hence, while all children with completed assessment 
or parent interview data have a known value of sex; there are children with completed school-level data whose sex is unknown. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 
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Table 6-11. Number of completed child-level cases and child- level completion rates for the student records abstract and teacher-level 
questionnaire for children sampled in the base year, by child characteristics: School year 2003-04 





Student records abstract 




Teacher-level questionnaire 






Completion rates 






Completion rates 


Child characteristics 1 


Completes 2 


Weighted Unweighted 


Completes^ 


Weighted 


Unweighted 


All children 


10,015 


69.9 


82.1 


10,872 


79.9 


90.6 


Sex 3 


Male 


5,034 


69.6 


81.6 


5,482 


79.3 


90.4 


Female 


4,981 


70.4 


82.7 


5,390 


80.7 


91.0 


Race/ethnicity 


White, non-Hispanic 


5,878 


72.6 


85.6 


6,300 


80.6 


92.3 


Black, non-Hispanic 


1,101 


63.9 


78.8 


1,238 


77.1 


90.8 


Hispanic 


1,769 


68.4 


75.5 


1,962 


80.2 


86.9 


Asian 


692 


70.7 


78.3 


751 


82.6 


88.2 


Pacific Islander 


125 


69.6 


79.6 


138 


84.0 


88.5 


American Indian or Alaska Native 


197 


66.0 


84.9 


206 


77.0 


91.6 


Other 


242 


72.8 


82.6 


263 


81.9 


91.3 


Unknown 


11 


30.5 


36.7 


14 


55.3 


50.0 


Y ear of birth 


1992 


2,996 


71.5 


83.7 


3,213 


80.1 


91.2 


1993 


6,976 


69.4 


81.7 


7,610 


80.1 


90.6 


Other/unknown 


43 


57.4 


55.8 


49 


54.4 


65.3 



1 Based on ECLS-K survey data and not on data from the sampling frame. 

2 A completed questionnaire was defined as one that was not left completely blank. 

3 The completion of the school-level instruments does not depend on whether the child has completed assessment or parent interview data. Hence, while all children with completed 
assessment and parent interview data have a known value of sex; there is one child with completed facilities checklist data whose sex is unknown. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 
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Table 6-12. 



Number of completed child-level cases and child-level completion rates for the child-level teacher questionnaires for children sampled in 
the base year, by child characteristics: School year 2003-04 





Child-level 

reading teacher questionnaire 




Child-level 

mathematics teacher questionnaire 


Child-level 

science teacher questionnaire 


Child characteristics 1 




Completion rates 






Completion rates 






Completion rates 




Completes 2 


Weighted Unweighted 


Completes 2 


Weighted Unweighted 


Completes 2 


Weighted Unweighted 


All children 


10,793 


79.3 


90.0 


5,339 


78.1 


89.3 


5,405 


79.5 


89.9 


Sex 3 


Male 


5,436 


78.7 


89.7 


2,670 


78.0 


88.9 


2,739 


78.2 


89.5 


Female 


5,357 


80.0 


90.5 


2,669 


78.4 


89.9 


2,666 


81.1 


90.4 


Race/ethnicity 


White, non-Hispanic 


6,255 


80.0 


91.7 


3,111 


78.6 


91.4 


3,147 


81.4 


92.0 


Black, non-Hispanic 


1,231 


76.7 


90.2 


591 


75.7 


88.5 


629 


76.2 


90.4 


Hispanic 


1,952 


79.5 


86.4 


959 


79.6 


85.2 


963 


77.2 


84.9 


Asian 


743 


81.8 


87.3 


362 


77.8 


85.8 


378 


85.1 


88.1 


Pacific Islander 


132 


79.1 


84.6 


75 


77.5 


85.2 


59 


81.1 


86.8 


American Indian or Alaska 


Native 


204 


76.4 


90.7 


109 


75.6 


91.6 


93 


77.0 


87.7 


Other 


262 


81.4 


91.0 


127 


80.2 


89.4 


129 


76.2 


88.4 


Unknown 


14 


55.3 


50.0 


5 


47.5 


35.7 


7 


31.2 


50.0 


Year of birth 


1992 


3,186 


79.1 


90.4 


1,565 


77.9 


90.1 


1,618 


79.9 


90.5 


1993 


7,559 


79.6 


90.0 


3,751 


78.4 


89.1 


3,762 


79.7 


89.8 


Other/unknown 


48 


53.5 


64.0 


23 


55.0 


63.9 


25 


52.5 


64.1 



1 Based on ECLS-K survey data and not on data from the sampling frame. 

2 A completed questionnaire was defined as one that was not left completely blank. 

3 There is a small number of children whose sex is unknown and who did not have completed teacher questionnaire data. The completion rates for these children, being zero, are not included in the table. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 



Table 6-13. Number of completed instruments and child-level completion rates for the special education 
teacher questionnaires for children sampled in the base year: School year 2003-04 



Category 


Completes 


Completion rates 
Weighted 


Unweighted 


Special education part A 1 


960 


92.2 


93.8 


Special education part B 1 


967 


93.7 


94.4 


A completed instrument was defined 


as one that was not left completely blank. 







SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



their completion rates are higher, 92.2 percent for part A, which captures teacher information, and 93.7 
percent for part B, which relates to children who receive individualized special education services. These 
rates are not broken down by school and child characteristics because of the small sample sizes. 



6.2.2 Children Sampled in First Grade 

In spring-first grade, the student sample was freshened to include first graders who had no 
chance of selection in the base year because they had not attended kindergarten in the United States or 
were in first grade in the fall of 1998. For a detailed description of the freshening procedure see chapter 3, 
section 3.4.2. This same group of children was followed into spring-fifth grade, unless they belonged in 
the excluded groups. Nonresponse in the freshened student sample could occur at two stages: during the 
procedure for sampling schools for freshening and identifying children to be used as freshening links in 
spring-first grade (first component) and then during data collection from the freshened children in spring- 
third grade (second component). 

The first component of the completion rate is the proportion of children sampled from the 
base year and subsampled for freshening for which the study was able to do freshening. The numerator 
includes all children available for freshening (i.e., those who did not move and in schools that cooperated 
with the freshening in first grade); the denominator includes children sampled for freshening (excluding 
movers not subsampled). For the weighted first component of the completion rate, both numerator and 
denominator were weighted by the product of the school base weight, the within-school child weight, and 
the freshening adjustment factor. The school base weight and the within-school child level weight reflect 
the multi-stage sampling of the ECLS-K design, while the freshening adjustment factor is necessary 
because schools were subsampled for freshening in first grade as described in section 3.4.2. These 
weights and adjustment factor are discussed in more detail in chapter 7. The first component alone can 
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further be decomposed into two sources: attrition due to the refusal of entire schools to implement the 
freshening procedure (the school term), and attrition due to ECLS-K sampled children moving to other 
schools (the child term). To contain costs, children who transferred from schools targeted for freshening 
were not used as links to identify freshened children, even when they were otherwise followed for data 
collection. These movers were considered freshening nonrespondents in the child term. 

The second component is the proportion of freshened children with completed data for the 
instrument from within the population brought into the sample by freshening. The weight for this 
component is the product of the school base weight, the within-school child weight, the school freshening 
subsample adjustment factor and the fifth-grade mover subsampling adjustment factor. 



The final completion rate is the product of the two components. For example, the final 
completion rate for the child assessment is computed as follows: 



r CA 



2 X H W 2i 

_ i^A ,, ieER CA 

IX- I W 2i 

i^B i eER CA kjENR ca 



where A is the set of children who could be freshened from (as described above), B is the set of children 
sampled for freshening, W fl is the weight for child i for the first component as described above, tf); is the 
weight for child i for the second component as described above, ER CA denotes eligible child assessment 
respondent, and ENRca eligible child assessment nonrespondent. To compute the unweighted rates, Wn 
and W 2 u are set to 1 for each child. 

Table 6-14 presents weighted and unweighted completion rates for freshened children. The 
two components of the completion rates are presented separately in table 6-14. The overall completion 
rates (i.e., the third set of rates in the table) are the products of the two components. The first component 
is separated into a school term and a child term as described earlier. For this component, the completion 
rate is defined as the freshening completion rates, as opposed to the survey instrument completion rates 
found in the second component. The weighted freshening completion rate for children in schools targeted 
for freshening ( the school term) is 67.2 percent. As part of the freshening process, schools were asked to 
prepare an alphabetic roster of students enrolled in first grade. These schools were also requested to 
identify which students did not attend kindergarten the previous year. Schools did not participate in the 
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Table 6-14. Number of completed child-level cases and child-level completion rates for children 
sampled in first grade: School year 2003-04 



Category 


Completes 


Completion rates 1 
Weighted Unweighted 


First component (first-grade sample freshening) 


7,032 


66.0 


79.1 


School term 2 


7,089 


67.2 


79.7 


Child term 3 


7,135 


98.2 


99.2 


Second component (fifth-grade data collection) 4 








Child assessment 5 


86 


78.6 


86.0 


Parent interview 6 


83 


81.9 


83.0 


School administrator questionnaire 7 


86 


74.4 


83.5 


Facilities check list 7 


89 


77.8 


86.4 


Student records abstract 7 


75 


67.0 


72.8 


Teacher- level questionnaire 7 


87 


81.2 


87.9 


Reading teacher questionnaire (child level) 7 


84 


79.8 


84.8 


Mathematics teacher questionnaire 


41 






(child level) 7 




81.0 


83.7 


Science teacher questionnaire (child level) 7 


40 


75.2 


80.0 


Special education part A 7 


15 


100.0 


100.0 


Special education part B 7 


14 


93.4 


93.3 


Overall completion rates 








Child assessment 5 


86 


51.9 


68.0 


Parent interview 6 


83 


54.0 


65.6 


School administrator questionnaire 7 


86 


49.1 


66.0 


Facilities check list 7 


89 


51.3 


68.3 


Student records abstract 7 


75 


44.2 


57.6 


Teacher-level questionnaire 7 


87 


53.6 


69.5 


Reading teacher questionnaire (child level) 7 


84 


52.7 


67.0 


Mathematics teacher questionnaire 


41 






(child level) 7 




53.5 


66.2 


Science teacher questionnaire (child level) 7 


40 


49.6 


63.2 


Special education part A 7 


15 


66.0 


79.1 


Special education part B 7 


14 


61.6 


73.8 



1 In the first component, this is the completion rate for freshening. In the second component, this is the completion rate for the survey instruments. 
The product of the two components is the overall completion rate for the survey instruments. 

2 The freshening completes and completion rates for children in schools targeted for freshening. 

3 The freshening completes and completion rates for children in schools that agreed to the freshening procedure. 

4 The number of children sampled via the freshening procedure was 165 but not all were eligible for the fifth-grade data collection. Some became 
ineligible by the time of the fifth-grade data collection, and some were movers not subsampled for followup. The denominator of the completion 
rate also varies according to the instrument. For example, children who were homeschooled were not eligible to have school and teacher 
questionnaires. The denominator for each instrument can be obtained by dividing the number of completes by the unweighted completion rate. 

5 Reading, mathematics, or science assessment was scorable, or child was disabled and could not be assessed. 

6 Family structure portion of parent interview was completed. 

7 A completed questionnaire was defined as one that was not completely left blank. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 
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freshening process because they either refused or were unable to provide the requested information. 
Within the schools that agreed to freshen, the freshening completion rate is 98.2 percent, the slight loss 
due to children who transferred to other schools ( the child term). Multiplying these two terms together 
gives a first component completion rate of 66.0 percent. Note that the first component rate for spring-fifth 
grade is not identical to the first component rate for spring-first grade and spring-third grade because of 
the exclusion of children in special groups as explained in section 3.6. 

The second component varies by survey instrument. The rates for the paper-and-pencil 
instruments range from 67.0 percent for the student records abstract to 100.0 percent for the special 
education questionnaire part A. The rate for the child assessment at 78.6 percent is about 6 points lower 
than for the kindergarten sample, and the parent interview, at 81.9 percent, is about 7 points lower. The 
rates for the school instruments and the student records abstract are also lower than for the kindergarten 
sample, but by a smaller amount. The rates for some of the teacher instruments are higher than for the 
kindergarten sample. The larger difference in the parent interview rates, as compared with the difference 
in the rates for the child assessment, is due to the different number of followups given the two groups of 
parents. For parents of children sampled in first grade, the fifth-grade data collection was the second 
followup, while most parents of children sampled in kindergarten had at least four followups (spring- 
kindergarten, spring-first grade, spring-third grade, and spring-fifth grade). The number of followups does 
not affect the completion rate of the child assessment in the same way since the child assessment was 
done in the school for most children and hence could be perceived as part of school activities. The final 
completion rate for each instrument is the product of the two components. Because of the low rates at the 
first stage, these range from a high of 53.6 percent for the teacher-level questionnaire to a low of 44.2 
percent for the student records abstract. 



6.2.3 Spring-Fifth Grade Completion Rates for All Children 

To compute the fifth-grade completion rate for the combined set of children sampled in the 
base year and children sampled in first grade, the fifth-grade completion rate for each group is weighted 
by the proportion of all children in that group, and the two weighted fifth-grade completion rates were 
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added together. For example, the weighted fifth-grade completion rate for the child assessment (CA) was 
computed as 



r CA - r CA,BY x 



Z Wi 

+ r CA 1 ST x 

£ W . LA ’ iS1 



Z W i 

iclST 

Z W i 



i&BYvlST i^BYvjlST 



where BY denotes base year, 1ST denotes first grade, r CAK y is the child assessment completion rate for 
children sampled in the base year, r C A,isr is the child assessment completion rate for children sampled in 
first grade, and W, is the final weight (C6CW0 for the child assessment) for child i. 

To get the weighted fifth-grade completion rate for the child assessment (which is 84.7 
percent for children sampled in the base year and 51.9 percent for children sampled in first grade), the 
weighted proportion of children who were sampled in the base year was 0.9762; the weighted proportion 
of children who were sampled in first grade was 0.0238. The fifth-grade weighted completion rate for the 
child assessment was 0.847x0.9762+0.519x0.0238=0.839, or 83.9 percent. 

Table 6-15 presents final spring-fifth grade completion rates for children sampled in 
kindergarten, children sampled in first grade, and all children combined. Because children sampled in first 
grade represent such a small fraction of the total population of children, their inclusion in the computation 
of the completion rate brings down the rates for all children by less than one percent relative to the rates 
for children sampled in kindergarten, even though the completion rates for children sampled in first grade 
are much lower than the kindergarten rates. The spring-fifth grade overall completion rates for the child 
assessment and the parent interview are 83.9 percent and 88.3 percent, respectively. These rates are 
higher than in third grade by about 4 percentage points for the child assessment and by about 1 1 
percentage points for the parent interview. In all tables, the unweighted completion rates are almost 
always higher than the weighted completion rates, by as much as 23 percent at the overall level. Where 
there is a large difference, it is due to movers who have larger weights and higher nonresponse rates than 
nonmovers. The weights of the movers were increased to account for the subsampling of movers. They 
also responded at a much lower rate than nonmovers, as shown earlier in tables 6-5 to 6-8. This difference 
is larger than in previous years because movers in fifth grade have much larger weights than in previous 
years (many more movers were not included in fifth grade, necessitating larger adjustment factors). Note 
that the unweighted completion rates follow the traditional ECLS-K pattern, that is, rates for the child 
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Table 6-15. Number of completed child-level cases and child-level completion rates, for children sampled in kindergarten and first grade, by 
survey instruments: School year 2003-04 





Children sampled in kindergarten 


Children sampled in 


first grade 




All children 






Completion rates 




Completion rates 




Completion rates 


Survey instrument 


Completes 


Weighted 


Unweighted 


Completes 


Weighted 


Unweighted 


Completes 


Weighted 


Unweighted 


Child assessment 1 


11,260 


84.7 


93.6 


86 


51.9 


68.0 


11,346 


83.9 


93.4 


Parent interview 2 


10,913 


89.1 


90.7 


83 


54.0 


65.6 


10,996 


88.3 


90.5 


School administrator 


questionnaire 3 


10,937 


77.1 


89.6 


86 


49.1 


66.0 


11,023 


76.4 


89.4 


Facilities check list 3 


11,154 


78.8 


91.4 


89 


51.3 


68.3 


11,243 


78.1 


91.2 


Student records abstract 3 


10,015 


69.9 


82.1 


75 


44.2 


57.6 


10,090 


69.3 


81.9 


Teacher-level questionnaire 3 


10,872 


79.9 


90.6 


87 


53.6 


69.5 


10,959 


79.3 


90.4 


Reading teacher 

questionnaire (child 


level) 3 


10,793 


79.3 


90.0 


84 


52.7 


67.0 


10,877 


78.7 


89.8 


Mathematics teacher 


questionnaire (child 
level) 3 


5,339 


78.1 


89.3 


41 


53.5 


66.2 


5,380 


77.5 


89.1 


Science teacher 

questionnaire (child 


level) 3 


5,405 


79.5 


89.9 


40 


49.6 


63.2 


5,445 


78.8 


89.7 


Special education part A ' 


960 


92.2 


93.8 


15 


66.0 


79.1 


975 


91.6 


93.7 


Special education part B 3 


967 


93.7 


94.4 


14 


61.6 


73.8 


981 


92.9 


94.2 



1 Reading, mathematics, or science assessment was scorable, or child was disabled and could not be assessed. 

2 Family structure portion of parent interview was completed. 

3 A completed questionnaire was defined as one that was not left completely blank. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 




assessment are higher than rates for the parent interview (93.4 percent for the child assessment and 90.5 
percent for the parent interview). This is again due to movers with large weights and to the fact that there 
are more parent-responding movers than child-responding movers. Thus, the weighted completion rates 
are higher for the parent interview than for the child assessment. 

Table 6-16 shows the completion rates for the child assessment, the parent interviews, and 
the school and teacher instruments for children who have nonzero child weights (C6CW0>0). These are 
children whose spring-fifth grade reading, mathematics, or science assessments were scorable, or children 
who could not be assessed because of disabilities. These conditioned completion rates are useful to 
analysts who want to assess the relationship between the different instruments in term of participation. 
The completion rates from the different instruments are dependent in that if data from one instrument are 
missing (e.g., parent instrument) it is likely that data from other instruments are also missing, (e.g., school 
administrator questionnaire). The conditioned completion rate for the child assessment is by definition 
100 percent. The rate slightly less than 100 percent, shown when children sampled in kindergarten are 
combined with children sampled in first grade, is due to the school freshening nonresponse for children 
sampled in first grade. 

When the completion rates are conditioned on the presence of the child weight, they are at 
least 1 3 points higher than the unconditional completion rates for all instruments but the parent interview 
and the special education questionnaires. For these last two instruments, the difference between the 
number of completes for the conditional and unconditional rates is very small; hence the conditional rates 
are not affected as much as for the other instruments. For the parent interview, the unconditional rate is 
fairly high for the reason explained earlier; that is, movers in fifth grade have much larger weights than in 
previous years and there are more parent-responding movers than child-responding movers (the weighted 
completion rates are higher for the parent interview than for the child assessment). This results in the 
smaller difference between conditional and unconditional of about 4 percent. For all the other 
instruments, the conditional completion rates are higher by 13.6 points for the child-level science teacher 
questionnaire and as high as 17.2 points for the facilities checklist. 
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Table 6-16. Number of completed child-level cases and child-level completion rates, for children with scorable reading, mathematics or science 
assessment or children not assessed due to disabilities, by survey instruments: School year 2003-04 





Children sampled in kindergarten 


Children sampled in 


first grade 




All children 






Completion rates 




Completion rates 




Completion rates 


Survey instrument 


Completes 


Weighted 


Unweighted 


Completes 


Weighted 


Unweighted 


Completes 


Weighted 


Unweighted 


Child assessment 1 


11,260 


100.0 


100.0 


86 


78.3 


86.8 


11,346 


99.5 


99.9 


Parent interview 2 


10,445 


92.4 


92.8 


77 


71.5 


77.6 


10,522 


91.9 


92.7 


School administrator 


questionnaire 3 


10,794 


93.4 


96.1 


83 


74.7 


83.7 


10,877 


93.0 


96.0 


Facilities check list 3 


11,015 


95.7 


98.0 


86 


78.3 


86.8 


11,101 


95.3 


97.9 


Student records abstract 3 


9,986 


85.6 


88.9 


74 


68.3 


74.6 


10,060 


85.2 


88.8 


Teacher-level questionnaire 3 


10,799 


93.4 


96.1 


84 


77.3 


84.8 


10,883 


93.0 


96.0 


Reading teacher 

questionnaire (child 


level) 3 


10,774 


93.3 


95.9 


84 


77.3 


84.8 


10,858 


92.9 


95.8 


Mathematics teacher 


questionnaire 
(child level) ' 


5,331 


92.7 


95.5 


41 


76.2 


82.7 


5,372 


92.3 


95.4 


Science teacher 


questionnaire (child 
level) 3 


5,394 


92.8 


95.4 


40 


74.9 


80.7 


5,434 


92.4 


95.3 


Special education part A' 


947 


93.0 


94.1 


14 


78.3 


86.8 


961 


92.6 


94.0 


Special education part B 3 


959 


95.1 


95.3 


14 


78.3 


86.8 


973 


94.7 


95.2 



1 Reading, mathematics, or science assessment was scorable, or child was disabled and could not be assessed. 

2 Family structure portion of parent interview was completed. 

3 A completed questionnaire was defined as one that was not left completely blank. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 



As explained in section 3.6, four groups of children were excluded from the fifth-grade data 
collection. These are (1) children who became ineligible in an earlier round (because they had died or 
moved out of the country), (2) children who were subsampled out in previous rounds because they had 
moved out of the original schools and were not subsampled to be followed, (3) children whose parents 
emphatically refused to cooperate (hard refusals) in any of the data collection rounds since spring- 
kindergarten, and (4) children eligible for the third-grade sample for whom there are neither first-grade 
nor third-grade data. Table 6-17 shows the completion rates for all instruments had children in the last 
two exclusion groups been counted as nonrespondents. These are children who would have been eligible 
for the fifth-grade collection but past experience showed that they would most likely be nonrespondents. 
When compared to table 6-15, the completion rates for all instruments in table 6-17 are lower, as 
expected, but only by about 2 percent, with the smallest difference for the student records abstract and 
largest difference for the parent interview. Note that the rates for mathematics and science teacher appear 
to be unchanged. Recall that only about half of the children had a mathematics teacher questionnaire 
filled out for them and the other half a science teacher questionnaire. Since the mathematics/science 
sampling flags were not assigned to children not included in the sample, we were not able to compute a 
correct completion rate for these two instruments for these children. But the pattern of completion rates 
would be the same as for the other instruments and we would expect a drop of about 2 percent for the 
mathematics and science teacher questionnaire. 



6.3 Unit Nonresponse Bias Analysis 

Among all the instruments administered in the fifth-grade data collection, the student records 
abstract (SRA) is the only instrument with an unweighted unit completion rate lower than 85 percent. The 
unweighted completion rate for this instrument is 82.1 percent for children sampled in kindergarten, 57.6 
percent for children sampled in first grade, for an overall unweighted response rate of 81.9 percent (see 
table 6-15). When these rates were computed for children with scorable reading, mathematics or science 
assessment or children not assessed due to disabilities, the unweighted completion rate is much higher 
(88.8 percent overall as shown in table 6-16). 

An analysis was undertaken for the SRA to examine the potential nonresponse bias that may 
exist for this instrument. Methods for nonresponse bias analysis in this situation include (1) comparisons 
of respondents and nonrespondents using available sample frame, (2) multivariate analysis to identify the 
characteristics of cases most likely to respond, and (3) comparison of respondents to known population 
characteristics from external sources. 
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Table 6-17. Number of completed child-level cases and child-level completion rates, if excluded children were fielded, by survey instruments: 
School year 2003-04 





Children sampled in kindergarten 


Children sampled in 


first grade 




All children 






Completion rates 




Completion rates 




Completion rates 


Survey instrument 


Completes 


Weighted 


Unweighted 


Completes 


Weighted 


Unweighted 


Completes 


Weighted 


Unweighted 


Child assessment 1 


11,260 


82.7 


86.7 


86 


47.6 


57.7 


11,346 


81.9 


86.5 


Parent interview 2 


10,913 


87.0 


84.0 


83 


49.7 


55.8 


10,996 


86.1 


83.8 


School administrator 


questionnaire 3 


10,937 


75.3 


83.3 


86 


45.2 


56.3 


11,023 


74.6 


83.1 


Facilities check list 3 


11,154 


77.0 


84.9 


89 


47.3 


58.2 


11,243 


76.3 


84.7 


Student records abstract 3 


10,015 


68.3 


76.3 


75 


40.7 


49.1 


10,090 


67.6 


76.1 


Teacher-level questionnaire 3 


10,872 


78.0 


84.1 


87 


49.2 


58.9 


10,959 


77.3 


83.9 


Reading teacher 

questionnaire (child 


level) 3 


10,793 


77.4 


83.5 


84 


48.3 


56.8 


10,877 


76.7 


83.3 


Mathematics teacher 


questionnaire 
(child level) ' 


5,339 


78.1 


89.3 


41 


52.1 


65.2 


5,380 


77.5 


89.1 


Science teacher 

questionnaire (child 


level) 3 


5,405 


79.5 


89.9 


40 


48.3 


62.3 


5,445 


78.8 


89.7 


Special education part A' 


960 


92.2 


93.8 


15 


64.3 


77.9 


975 


91.5 


93.7 


Special education part B 3 


967 


93.7 


94.4 


14 


60.0 


72.7 


981 


92.9 


94.2 



1 Reading, mathematics, or science assessment was scorable, or child was disabled and could not be assessed. 

2 Family structure portion of parent interview was completed. 

3 A completed questionnaire was defined as one that was not left completely blank. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 




There are 45 items in the SRA and all but seven are suppressed in the public-release data 
file. The suppressed items are dates when the child entered and left school, whether school kept 
attendance record, presence of IEP records, and items on learning disabilities. The seven items that 
are not suppressed are items on absence and tardiness. Because there are no school frames that report 
this type of data and there are no external sources that include these data, the nonresponse bias 
analysis is limited to analyzing the relationship between the SRA response status and response 
indicators such as key school characteristics. The Chi-square Automatic Interaction Detector 
(CHAID) was used to look at the relationship on variables with known values for both respondents 
and nonrespondents. CHAID is a classification algorithm that uses chi-square tests to divide the 
sample into subgroups that are related to whether the unit responds. 

The analysis in CHAID begins by dividing the sample into two or more groups based on 
the categories of the best predictor of response. Each of these groups is divided into smaller 
subgroups based on the available predictors at each level. The splitting process continues until no 
statistically significant predictor remains. The CHAID software displays the final subgroups in the 
form of a tree diagram whose branches correspond to the groups, showing all potential response 
predictors. The resulting classification tree reveals the response cells, as defined by combination of 
variables, which identify cells with the lowest response rate. In other words, CHAID divides the 
sample into cells so that the response rate within cells is as constant as possible, while the response 
rate between cells is as different as possible. 

The response indicators used in CHAID are school characteristics that are known for the 
respondents and nonrespondents to the SRA: census region, school affiliation, type of locale, total 
enrollment, and percent non- White enrolled. Exhibit 6-1 shows the CHAID tree when the SRA 
response status is analyzed together with these response indicators. Each box represents a group of 
SRAs with specified characteristics, the weighted completion rate and the number of fifth graders 
with completed SRA (i.e., SRA respondents). The first branch of the tree shows that census region is 
the first indicator that was used to divide the SRAs into four groups with response patterns that are 
very different from each other (with significance level of 0.05). The response rates in the four groups 
are 71.5 in the Northeast, 89.7 percent in the Midwest, 84.4 percent in the South, and 80.7 percent in 
the West. In the next branch of the tree, SRAs in the Northeast could be further split into four groups 
with very different response pattern by percent of non- White enrolled in the school, while those in the 
Midwest were subdivided into two groups with differential response patterns based on the type of 
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locale. Thus, the tree continues and ends after the sample was divided into 49 cells with varying 
response patterns. 

All variables used in the analysis show up in the tree as significant predictors of response 
patterns. Some are more prevalent than others. For example, school affiliation is not a significant 
indicator of response in the South while the total school enrollment is a significant predictor of 
response for all instances in the South. School affiliation is a significant response indicator in two 
groups of SRAs in the Midwest (SRAs from schools in large and mid-size cities/suburb of large and 
mid-size cities/large towns where the percent of non- White enrolled is less than 50 percent, and where 
the total school enrollment is less than 750; and, SRAs from schools in small towns and rural areas 
where the percent of non- White enrolled is less than 50 percent), and in the West (where schools have 
11 or more but less than 50 percent of non- White enrolled in large and mid-size cities, small towns 
and rural areas, and where schools have 50 or more but less than 90 percent of non- White enrolled). 
School affiliation is significant in only one group in the Northeast (where schools have 10 percent or 
less non- White enrolled). The other variables (total school enrollment, percent non- White enrolled, 
and type of locale) are all good predictors of response. The lower completion rates are in the 
Northeast, in schools that are nonreligious private and in schools with high enrollment 

Potential nonresponse bias that may exist is likely to have been lessened by the 
weighting adjustment procedures described in chapter 7. School affiliation, type of locale and region 
were used as raking dimensions in the last step of the weighting procedures for all weights including 
the child-level weight C6CW0, the most appropriate weight to use in the analysis of the SRA data. 



6.4 Item Response Rates 

In the ECLS-K, as in most surveys, the responses to some data items are not obtained for 
all interviews. There are numerous reasons for item nonresponse. Some respondents do not know the 
answer for the item or do not wish to respond for other reasons. Some item nonresponse arises when 
an interview is interrupted and not continued later, leaving items at the end of the interview blank. 
Item nonresponse may also be encountered because responses provided by the respondent are not 
internally consistent, and this inconsistency is not discovered until after the interview is completed. In 
these cases, the items that were not internally consistent were set to missing. 
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Exhibit 6-1. Relationship between the student record abstract response status and school 
characteristics: School year 2003-04 



Census region 



Northeast 



71.5% 

(1,647) 



Percent nonwhite 
enrolled 



0-10 



78.1% 



School affiliation 



(56) 





(903) 


Public 


79.1% 


Total enrolled 










(712) 


1-499 


75.0% 


Type of locale 












(456) 


City/suburb/ 


66.8% 














large town 


(173) 














Small town/ 


83.0% 














rural 


(283) 










>=500 


85.9% 


Cell 3 














(256) 










Catholic 


82.2% 


Cell 4 














(156) 














Other 


48.4% 


Cell 5 












private 


(35) 










11-49 


63.6% 


Type of locale 












(334) 


City 


87.7% 


Total enrolled 












(95) 


1-749 


83.9% 


Cell 6 














(65) 














>=750 


100.0% 


Cell 7 














(30) 










Suburb/ 


57.7% 


Cell 8 












town/rural 


(239) 










50-89 


83.9% 


Total enrolled 












(243) 


1-499 


92.9% 


Cell 9 














(124) 














>=500 


78.1 % 


Cell 10 














(119) 










90-100 


52.6% 


Total enrolled 












(167) 


1-299 


76.0% 


Cell 11 














(31) 














300-749 


49.1% 


Cell 12 














(80) 














>=700 


49.6% 


Cell 13 









See note at end of exhibit. 



Cell 1 
Cell 2 
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Exhibit 6- 1 . Relationship between the student record abstract response status and school 
characteristics: School year 2003-04 — Continued 



Census region 
Midwest 89.7% 

(2,799) 



See note at end of exhibit. 



Type of locale 



City/suburb/ 86.5% 

large town (1,752) 



Small town/ 96.0% 

rural (1,047) 



Percent nonwhite 
enrolled 



(175) 



0-49 


90.1% 


Total enrolled 






(1,432) 


1-749 


89.2% 


School affiliation 








(1,277) 


Public 


87.6% 












(867) 










Private 


94.9% 












(410) 






>=750 


99.7% 


Cell 16 










(155) 






50-89 


82.2% 


Total enrolled 








(222) 


1-499 


76.7% 


Cell 17 










(150) 










>=500 


96.0% 


Cell 18 










(72) 






90-100 


67.2% 


Cell 19 










(98) 










Percent nonwhite 












enrolled 










0-49 


95.9% 


School affiliation 








(872) 


Public 


97.5% 


Cell 20 










(722) 










Catholic 


97.1% 


Cell 21 










(117) 










Other 


65.6% 


Cell 22 








private 


(33) 






50-100 


96.9% 


Cell 23 









Cell 14 
Cell 15 
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Exhibit 6- 1 . Relationship between the student record abstract response status and school 
characteristics: School year 2003-04 — Continued 



Census region 



South 84.4% 

(3,401) 



Percent nonwhite 
enrolled 










0-10 


91.5% 

(782) 


Total enrolled 


Cell 24 


1-299 


75.5% 

(125) 


>=300 


94.4% 

(657) 


Type of locale 


City / 
unknown 


98.8% 

(151) 


Suburb/town/ 

rural 


93.6% 

(506) 


11-49 


83.7% 


Total enrolled 






(1,345) 


1-299 


96.8% 


Type of locale 








(227) 


City/suburb/ 


94.1% 










large town 


(125) 










Small town/ 


100.0% 










rural 


(102) 






>=300 


82.1% 


Cell 29 










(1,118) 






50-89 


81.4% 


Total enrolled 








(602) 


1-749 


86.4% 


Type of locale 








(481) 


City/suburb/ 


84.3% 










large town 


(392) 










Small town/ 


96.6% 










rural 


(89) 






>=750 


64.6% 


Cell 32 










(121) 






90-100/ 


82.2% 


Total enrolled 






unknown 


(672) 


1-299 


71.2% 


Cell 33 










(71) 










300-499 


94.4% 


Type of locale 








(250) 


City/suburb/ 


93.6% 










large town 


(200) 










Small town/ 


100.0% 










rural 


(50) 






500-749 


86.1% 


Cell 36 










(219) 










>=750 


68.3% 


Cell 37 










(132) 







See note at end of exhibit. 



Cell 25 
Cell 26 

Cell 27 
Cell 28 



Cell 30 
Cell 31 



Cell 34 
Cell 35 
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Exhibit 6- 1 . Relationship between the student record abstract response status and school 
characteristics: School year 2003-04 — Continued 



Census region 



West 



80.7% 

(2,243) 



Percent nonwhite 
enrolled 



0-10 



98.9% 

1160) 



11-49 



82.9% 

(788) 



50-89 



74.5% 

(611) 



90-100 



78.9% 

(684) 



Cell 38 



Type of locale 



City/ 


89.6% 


School affiliation 




small town/ 


(488) 


Public 


90.6% 


Total enrolled 


rural 






(305) 


1-299 


100.0% 












(54) 










300-499 


85.8% 












(121) 










>=500 


92.6% 












(130) 






Catholic 


99.4% 


Cell 42 










(122) 










Other 


74.1% 


Cell 43 








private 


(61) 






Suburb/ 


75.6% 


Total enrolled 






large town 


(300) 


1-499 


84.4% 


Cell 44 










(167) 










>=500 


67.5% 


Cell 45 










(133) 






School affiliation 










Public 


74.2% 


Cell 46 










(515) 










Catholic/ 


79.5% 


Type of locale 






other private 


(96) 


City 


97.5% 


Cell 47 










(50) 










Suburb/ 


65.7% 


Cell 48 








Town/ 


(46) 










Rural 









Cell 49 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten 
Class of 1998-99 (ECLS-K), spring 2004. 



Cell 39 
Cell 40 
Cell 41 
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Every item in the ECLS-K data file has values that indicate whether the respondent did not know the 
answer to the item (-8), or refused to give an answer (-7). The value -9 is used in all other cases where 
the answer is left blank or set to missing due to reasons mentioned above (described in the data file 
as “Not ascertained”). However, where an item is left bla nk due to a valid skip pattern, this is 
indicated by the value -1. Chapter 7 of the ECLS-K Combined User’s Manual for the ECLS-K Fifth 
Grade Public-Use Data Files and Electronic Codebooks. (NCES 2006-032) (Tourangeau et al. 
forthcoming) discusses in detail these special values. For each survey item, the response rate was 
computed as the unweighted number of responses not equal to any of the special values (-1, -7, -8, 
or -9) divided by the unweighted number of responses not equal to -1. Of all the ECLS-K 
instruments, only the child assessment and the parent interview have a sizable number of items with 
special values -7 (“Refused”) or -8 (“Don’t know”). Table 6-18 shows the unweighted distribution of 
the nonresponse values for each instrument. 



Table 6-18. Number of survey items and percent of nonresponse values: School year 2003-04 



Survey instrument 


Number of 
items 


Items with nonresponse value (in percent) 

-7 -8 -9 

(Refused) (Don’t know) (Not ascertained) 


Total 


1,758 


17.8 


24.3 


77.9 


Child assessment 


85 


21.2 


21.2 


76.5 


Parent interview 


786 


37.5 


48.7 


55.2 


School administrator questionnaire 


215 


0.0 


5.6 


98.6 


Facilities checklist 


31 


0.0 


0.0 


96.8 


Student records abstract 


43 


0.0 


0.0 


95.3 


Teacher-level questionnaire 


183 


0.0 


3.3 


99.5 


Child-level reading teacher questionnaire 


107 


0.0 


0.0 


99.1 


Child-level mathematics teacher questionnaire 


75 


0.0 


4.0 


98.7 


Child-level science teacher questionnaire 


62 


0.0 


8.1 


96.8 


Special education part A 


53 


0.0 


0.0 


92.5 


Special education part B 


119 


0.0 


0.8 


98.3 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten 
Class of 1998-99 (ECLS-K), spring 2004. 



For most of the data items collected in the ECLS-K, the item response rate unweighted 
was very high, with an overall median item response rate of 98.8 percent. The median item response 
rate for each of the instruments ranges from 91.2 for the student records abstract to 99.8 for the child 
assessment. Table 6-19 shows the number of items, the median item response rate, the lowest item 
response rate, the highest item response rate, and the number of items with response rates of less than 
85 percent for all instruments. Items with less than 85 percent response rates are listed in table 6-20 
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by instrument and in ascending order of item response rate. The number of cases for which each item 
was attempted is also shown in this table. The tables in this chapter show the item response rates for 
items on the restricted-use file. Rates in tables 6-19 and 6-20 are unweighted. 



Table 6-19. Item response rates for items on the ECLS-K fifth-grade restricted-use data file: School 
year 2003-04 



Instmment 


Number of 
items 


Median 

response 

rate 

(percent) 


Lowest 

response 

rate 

(percent) 


Highest 

response 

rate 

(percent) 


Number of 
items with 
response rate 
< 85 percent 


Total 


1,758 


98.8 


0.0 


100.0 


163 


Child assessment data 


85 


99.8 


93.3 


100.0 


0 


Parent interview 


786 


99.6 


0.0 


100.0 


96 


School administrator questionnaire 


215 


98.0 


49.1 


100.0 


33 


Facilities checklist 


31 


95.2 


34.7 


100.0 


12 


Student records abstract 


43 


91.2 


43.9 


100.0 


14 


Teacher-level questionnaire 


183 


98.2 


63.5 


100.0 


5 


Child-level reading teacher questionnaire 


107 


98.6 


89.3 


100.0 


0 


Child-level mathematics teacher questionnaire 


75 


99.3 


83.3 


100.0 


1 


Child-level science teacher questionnaire 


62 


99.0 


93.4 


100.0 


0 


Special education part A 


53 


97.0 


92.3 


100.0 


0 


Special education part B 


119 


98.5 


76.6 


100.0 


2 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten 
Class of 1998-99 (ECLS-K), spring 2004. 
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Table 6-20. Items on the ECLS-K fifth-grade restricted-use data file with less than 85 percent 
response rates: School year 2003-04 



Variable name 


Description 


Number 

eligible 


Item response rate 
(percent) 


Parent interview 








P6DGNELE 


P6 CHQ137 DIAGNOSIS MADE BEFORE ELEM SCH 


59 


0.0 


P6DGHR02 


P6 CHQ226 DIAGNOSIS MADE BEFORE 2002 


60 


0.0 


P6DGNELS 


P6 CHQ227 DIAGNOSIS MADE BEFORE ELEM SCH 


60 


0.0 


P6IMPT02 


P6 CHQ252 WAS IT BEFORE 2002 


60 


0.0 


P6IMPELM 


P6 CHQ253 WAS IT BEFORE ELEM SCHOOL 


60 


0.0 


P6DNBF02 


P6 CHQ346 DIAGNOSIS MADE BEFORE 2002 


61 


0.0 


P6BELDGN 


P6 CHQ347 DIAGNOSIS MADE BEFORE ELEM SCH 


61 


0.0 


P6SERVRV 


P6 CHQ536 SERVICES RCVD BEFORE 2002 


66 


0.0 


P6DGSYR 


P6 CHQ376 DIAGNOSIS MADE BEFORE 2002 


62 


1.6 


P6DGSESC 


P6 CHQ377 DIAGNOSIS MADE BEFORE ELEM SCH 


62 


1.6 


P6DGN02 


P6 CHQ136 DIAGNOSIS MADE BEFORE 2002 


60 


1.7 


P6SRVRCV 


P6 CHQ537 SRVCS RCVD BEFORE ELEM SCHOOL 


66 


3.0 


P6DGSP02 


P6 CHQ186 DIAGNOSIS MADE BEFORE 2002 


63 


4.8 


P6ELMDGN 


P6 CHQ187 DIAGNOSIS MADE BEFORE ELEM SCH 


63 


4.8 


P6REL2 


P6 FSQ130 2ND PERSON’S RELATION TYPE 


10,996 


5.8 


P6CHIN1 


P6 NRQ050 CONTACT W/BIOMOM SAME AS CHD1 


67 


7.5 


P6CHIN3 


P6 NRQ050 CONTACT W/BIODAD SAME AS CHD 1 


193 


9.8 


P6DGBF02 


P6 CHQ314 DIAGNOSIS MADE BEFORE 2002 


71 


12.7 


P6IMPLNT 


P6 CHQ251 YEAR OF IMPLANT 


69 


13.0 


P6CLRUSE 


P6 CHQ254 USE OF COCHLEAR IMPLANT IN SCH 


69 


13.0 


P6DGBFEL 


P6 CHQ315 DIAGNOSIS MADE BEFORE ELEM SCH 


71 


15.5 


P6DIAG02 


P6 CHQ076 DIAGNOSIS MADE BEFORE 2002 


73 


17.8 


P6DIAGEF 


P6 CHQ077 DIAGNOSIS MADE BEFORE ELEM SCH 


73 


17.8 


P6HEARS2 


P6 CHQ260 DEVICE EFFECT ON CHD’S HEARING 


76 


21.1 


P6AGREE3 


P6 NRQ264 AGREEMENT W/ ADOPTIVE FATHER 


282 


38.3 


P6HEARS 


P6 CHQ230 DEGREE OF CHILD’S DEAFNESS 


111 


39.6 


P60FTCUT 


P6 FDQ230 FREQ CHILD SKIP MEAL -NO FOOD 


131 


42.7 


P6NOTEA2 


P6 FDQ200 FREQ NOT EAT ENTIRE DAY 


138 


45.7 


P6YYDIA5 


P6 CHQ225 YR AT 1ST DIAGNOSIS-HEARING 


111 


45.9 


P6HEARAI 


P6 CHQ240 IF CHILD WEARS HEARING AID 


111 


45.9 


P6COCHEE 


P6 CHQ250 IF CHILD HAS COCHLEAR IMPLANTS 


111 


45.9 


P6ERSRVY 


P6 CHQ535 YR LAST RECEIVED SERVICES 


152 


56.6 


P6MTHAGE 


P6 COQ010 AGE OF MOTHR WHEN MOVED TO U S 


213 


56.8 


P6 AGREE 1 


P6 NRQ264 AGREEMENT W/ BIOLOGICAL FATHER 


145 


57.2 


P6NHROTH 


P6 CCQ250 # HRS/WK RECVS CARE OTHER NREL 


95 


57.9 


P6CHROTH 


P6 CCQ403 # HRS/WK AT OTHER PROGRAMS 


105 


58.1 


P6SCHSA 


P6 WPQ106 ATTEND SCHOOL -SA 


187 


58.8 


P6LKSA 


P6 WPQ106 LOOK FOR WORK - SA 


187 


58.8 


P60THSA 


P6 WPQ106 OTHER - SA 


187 


58.8 


P6UNJBSA 


P6 WPQ106 WORK FOR NO PAY - SA 


187 


58.8 


P6PDJBSA 


P6 WPQ106 WORK FOR PAY - SA 


187 


58.8 


P6DIFFH3 


P6 CHQ210 IF HEAR DIFFICULTY DIAGNOSED 


149 


59.1 



See note at end of table. 
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Table 6-20. Items on the ECLS-K fifth-grade restricted-use data file with less than 85 percent 
response rates: School year 2003-04 — Continued 



Variable name 


Description 


Number 

eligible 


Item response rate 
(percent) 


Parent interview (continued) 






P6FTHAGE 


P6 COQ025 AGE OF FATHR WHEN MOVED TO U S 


619 


62.0 


P6DIFFH2 


P6 CHQ200 IF HEAR DIFFICULTY EVALUATED 


185 


66.5 


P6FMTHRS 


P6 CHQ780 REASON FOR FAMILY THERAPY 


237 


68.8 


P6SEEHLP 


P6 CHQ800 TIMES FAMILY SAW HELP 


237 


68.8 


P6SEEPSY 


P6 CHQ790A FAMILY SEE A PSYCHIATRIST 


237 


69.2 


P6SEPSYC 


P6 CHQ790B FAMILY SEE A PSYCHOLOGIST 


237 


69.2 


P6SEESOC 


P6 CHQ790C FAMILY SEE A SOCIAL WORKER 


237 


69.2 


P6SEECSL 


P6 CHQ790D FAMILY SEE A COUNSELOR 


237 


69.2 


P6SEESOM 


P6 CHQ790E FAMILY SEE SOMEONE ELSE 


237 


69.2 


P6MTHCTZ 


P6 COQ015 IS MOTHER AU S CITIZEN 


213 


70.0 


P6FPT2 


P6 PEQ040 PERS 2 COURSE FULL/PART TIME 


211 


70.6 


P6DGNACT 


P6 CHQ120 WHAT 1ST DIAGNOSIS - ACTIVITY 


226 


72.6 


P6YYDIA2 


P6 CHQ135 YR AT 1ST DIAGNOSIS-ACTIVITY 


226 


73.5 


P6TAK2 


P6 EMQ100 PERS 2 JOB AVAILABLE LAST WEEK 


234 


73.5 


P6D032 


P6 EMQ070 PERS 2 CHKD W/EMPLOYR DIRECTLY 


248 


73.8 


P6D042 


P6 EMQ070 PERS 2 CHKD W/FRIENDS & REL 


248 


73.8 


P6D022 


P6 EMQ070 PERS 2 CHKD W/PRIV EMP AGNCY 


248 


73.8 


P6D012 


P6 EMQ070 PERS 2 CHKD W/PUB EMPL AGNCY 


248 


73.8 


P6D072 


P6 EMQ070 PERS 2 DID SOMETHING ELSE 


248 


73.8 


P6D052 


P6 EMQ070 PERS 2 PLACED OR ANSWERED ADS 


248 


73.8 


P6D062 


P6 EMQ070 PERS 2 READ WANT ADS 


248 


73.8 


P6BESTEY 


P6 CHQ320 WHAT CAN CHILD BEST SEE 


372 


74.5 


P6DGNBEH 


P6 CHQ337 1ST DIAGNOSIS-BEHAVIOR 


244 


74.6 


P6DGBEYY 


P6 CHQ345 YR AT 1ST DIAGNOSIS-BEHAVIOR 


244 


75.0 


P6TINCTH 


P6 PAQ120 TOTAL HOUSEHOLD INCOME ($-LOW) 


1,816 


75.7 


P6SCHFS 


P6 WPQ130 ATTEND SCHOOL -FS 


327 


76.5 


P6LKFS 


P6 WPQ130 LOOK FOR WORK - FS 


327 


76.5 


P60THFS 


P6 WPQ130 OTHER -FS 


327 


76.5 


P6UNJBFS 


P6 WPQ 1 30 WORK FOR NO PAY - FS 


327 


76.5 


P6PDJBFS 


P6 WPQ 130 WORK FOR PAY - FS 


327 


76.5 


P6TAK1 


P6 EMQ100 PERS 1 JOB AVAILABLE LAST WEEK 


449 


77.7 


P6CCNMCH 


P6 CCQ400 CC # CHILDREN AMOUNT FOR 


194 


78.4 


P6RCNMCH 


P6 CCQ135 RC # CHILDREN AMOUNT FOR 


182 


79.1 


P6DIABEH 


P6 CHQ335 BEHAVIOR PROBLEM DIAGNOSED 


297 


79.5 


P6D031 


P6 EMQ070 PERS 1 CHKD W/EMPLOYR DIRECTLY 


488 


79.7 


P6D041 


P6 EMQ070 PERS 1 CHKD W/FRIENDS & REL 


488 


79.7 


P6D021 


P6 EMQ070 PERS 1 CHKD W/PRIV EMP AGNCY 


488 


79.7 


P6D011 


P6 EMQ070 PERS 1 CHKD W/PUB EMPL AGNCY 


488 


79.7 


P6D071 


P6 EMQ070 PERS 1 DID SOMETHING ELSE 


488 


79.7 


P6D05 1 


P6 EMQ070 PERS 1 PLACED OR ANSWERED ADS 


488 


79.7 



See note at end of table. 
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Table 6-20. Items on the ECLS-K fifth-grade restricted-use data file with less than 85 percent 
response rates: School year 2003-04 — Continued 







Number 


Item response rate 


Variable name 


Description 


eligible 


(percent) 


Parent interview (continued) 






P6D061 


P6 EMQ070 PERS 1 READ WANT ADS 


488 


79.7 


P6PROFFD 


P6 CHQ1 10 IF ACTIVITY PROBLEM DIAGNOSED 
P6 COQ005 COUNTRY BIOLOGCAL MTHR WAS 


301 


80.4 


P6MTHBTH 


BRN 


1,273 


80.5 


P6FTHCTZ 


P6 COQ030 IS FATHER A U S CITIZEN 


619 


80.6 


P6FPT1 


P6 PEQ040 PERS 1 COURSE FULL/PART TIME 


498 


80.7 


P6RCCOST 


P6 CCQ125 COST OF RELATIVE CARE 


301 


82.1 


P6NCNMCH 


P6 CCQ240 NC # CHILDREN AMOUNT FOR 


219 


82.2 


P6YYDIA4 


P6 CHQ185 YEAR AT 1ST DIAGNOSIS-SPEECH 


373 


83.1 


P6DGNEMO 


P6 CHQ365 1ST DIAGNOSIS-EMOTIONAL BEH 


424 


83.3 


P6REQSA 


P6 WPQ105 ANY RQ FOR STATE AID (SA) 


491 


83.9 


P6WHENAF 


P6 WPQ102 # MNTHS RECEIVED TANF/AFDC 


491 


84.1 


P6MIN_N 1 


P6 NRQ250 BIOMOM LIVES # MINUTES AWAY 


641 


84.2 


P6TWNCAR 


P6 CCQ003 TWIN SAME CARE AS CHILD 


236 


84.7 


P6EVCUT 


P6 FDQ150 FREQ CUT MEAL SIZE 


488 


84.8 


School administrator questionnaire 






S6RTCHPT 


S6 Q38A2 # REG CLASSROOM TCHR-PART 


10,916 


49.1 


S6READPT 


S6 Q38E2 # READING TCHR/SPECIALIST-PART 


10,916 


54.2 


S6LIBRPT 


S6 Q38J2 # LIBRARIANS-PART 


10,916 


54.7 


S6SPEDPT 


S6 Q38C2 # SPECIAL ED TCHR-PART 


10,916 


56.8 


S6PARAPT 


S6 Q38I2 # PARAPROFESSIONALS-PART 


10,916 


57.9 


S6ESLPT 


S6 Q38D2 # ESL/BILINGUAL TCHR-PART 


10,916 


58.5 


S6PRABRK 


S6 Q19A2 PARTICIPATE ANY SCH BREAKFAST 


7,378 


58.8 


S6PAALUN 


S6 Q20A2 PARTICIPATE ANY SCH LUNCH 


10,916 


60.6 


S6GIFTPT 


S6 Q38F2 # GIFTED/TALENTED TCHR-PART 


10,916 


60.7 


S6NURSPT 


S6 Q38G2 # SCH NURSE HEALTH PROF-PART 


10,916 


65.2 


S6PSYCFL 


S6 Q38H1 SCH PSYCH/ SOCIAL WORKER-FULL 


10,916 


65.6 


S6MSARPT 


S6 Q38B2 # GYM DRAMA MUSIC ART TCHR-PART 


10,916 


67.4 


S60THER 


S6 Q15F SCH HAS OTHER REASON 


3,499 


68.0 


S6GIFTFL 


S6 Q38F1 # GIFTED/TALENTED TCHR-FULL 


10,916 


72.4 


S6NURSFL 


S6 Q38G1 # SCH NURSE HEALTH PROF-FULL 


10,916 


73.1 


S6PSYCPT 


S6 Q38H2 SCH PSYCH/SOCIAL WORKER-PART 


10,916 


74.0 


S6ELRPBK 


S6 Q19C1 ELIGIBLE RED-PRICE BREAKFAST 


7,378 


74.2 


S6ELIBRK 


S6 Q19B1 ELIGIBLE FOR FREE BREAKFAST 


7,378 


74.5 


S6ELIRED 


S6 Q20C1 ELIGIBLE IN REDUCED-PRICE LUNCH 


10,916 


75.9 


S6ESLFL 


S6 Q38D1 # ESL/BILINGUAL TCHR-FULL 


10,916 


76.5 


S6ELILNC 


S6 Q20B1 ELIGIBLE FOR FREE LUNCH 


10,916 


76.6 


S6PCTMTH 


S6 Q35B MATHEMATICSEMATICS SKILLS % 


10,625 


78.5 


S6PARPBK 


S6 Q19C2 PARTICIPATE RED-PRICE BREAKFAST 


7,378 


78.6 


S6PCTRD 


S6 Q35A READING OR VERBAL SKILLS % 


10,625 


79.2 


S6HWLONG 


S6 Q39E RESP (NOT PRINCIPAL) YR AT SCH 


2,607 


79.4 



See note at end of table. 
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Table 6-20. Items on the ECLS-K fifth-grade restricted-use data file with less than 85 percent 
response rates: School year 2003-04 — Continued 



Variable name 


Description 


Number 

eligible 


Item response rate 
(percent) 


School administrator questionnaire (continued) 






S6PARBRK 




S6 Q19B2 PARTICIPATES IN BREAKFAST 


7,378 


81.7 


S6AMBUSL 




S6 Q12 TIME LAST BUS AM 


10,916 


82.0 


S6MSARFL 




S6 Q38B1 # GYM DRAMA MUSIC ART TCHR-FULL 


10,916 


82.7 


S6PARRED 




S6 Q20C2 PARTICIPATES IN RED-PRICE LUNCH 


10,916 


83.2 


S6PARAFL 




S6 Q38I1 # PARAPROFESSIONALS-FULL 


10,916 


83.8 


S6LIBRFL 




S6 Q38J1 # LIBRARI ANS-FULL 


10,916 


84.0 


S6PARLNC 




S6 Q20B2 PARTICIPATES IN FREE LUNCH 


10,916 


84.5 


S6READFL 




S6 Q38E1 # READING TCHR/SPECIALIST-FULL 


10,916 


85.0 


Facilities checklist 








K6SEXITS 




K6 Q1E2 SCH CNFRM SECURED EXIT DOORS 


3,485 


34.7 


K6FENCES 




K6 Q1F2 SCH CNFRM FENCING AROUND SCHOOL 


4,669 


44.2 


K6BARSS 




K6 Q1D2 SCH CNFRM WINDOW AND DOOR BARS 


9,596 


45.4 


K6METDTS 




K6 Q1B2 SCH CNFRM METAL DETECTORS 


10,568 


48.2 


K6GUARDS 




K6 Q1A2 SCH CNFRM SECURITY GUARD 


9,902 


48.2 


K6SIGNOS 




K6 Q1H2 SCH CNFRM SIGN-IN AT OFFICE 


3,083 


48.6 


K6SCAMS 




K6 Q1C2 SCH CNFRM SECURITY CAMERAS 


8,465 


49.2 


K6SIGNPS 




K6 Q1G2 SCH CNFRM SIGN-IN POLICIES 


874 


53.5 


K6SPRNKS 




K6 Q1L2 SCH CNFRM FIRE SPRINKLERS 


5,357 


54.2 


K6FREXTS 




K6 Q1K2 SCH CNFRM FIRE EXTINGUISHERS 


287 


63.8 


K6INTCMS 




K6 Q1I2 SCH CNFRM INTERCOMS 


578 


70.1 


K6ALARMS 




K6 Q1J2 SCH CNFRM FIRE ALARMS 


183 


71.6 


Student records abstract 






U6WHYLFT 




U6 Q4 WHY CHILD LEFT SCHOOL 


171 


43.9 


U6MMLEFT 




U6 Q3A MONTH LEFT SCHOOL 


171 


62.0 


U6DDLEFT 




U6 Q3B DAY LEFT SCHOOL 


171 


62.0 


U6YYLEFT 




U6 Q3C YEAR LEFT SCHOOL 


171 


62.0 


U6YY01 




U6 Q9C3 YEAR 2001-2002 IEP SIGNED 


885 


63.8 


U6MM01 




U6 Q9C2 MONTH 2001-2002 IEP SIGNED 


885 


64.2 


U6YY02 




U6 Q9B3 YEAR 2002-2003 IEP SIGNED 


947 


71.0 


U6MM02 




U6 Q9B2 MONTH 2002-2003 IEP SIGNED 


947 


71.3 


U6IEP01 




U6 Q9C1 PRESENCE OF 2001-2002 IEP RECORD 


1,081 


75.2 


U6IEP02 




U6 Q9B1 PRESENCE OF 2002-2003 IEP RECORD 


1,081 


80.9 


U6YY03 




U6 Q9A3 YEAR 2003-2004 IEP SIGNED 


1,049 


82.4 


U6MM03 




U6 Q9A2 MONTH 2003-2004 IEP SIGNED 


1,049 


82.9 


U6BUXTAR 




U6 Q7C CHILD UNEXCUSED TARDIES 


10,064 


83.7 


U6BXTARD 




U6 Q7B CHILD EXCUSED TARDIES 


10,064 


83.9 


Teacher-level questionnaire 






J61TXFOR 




J61 Q1H2 TIME FOR FOREIGN LANGUAGE 


3,129 


63.5 


J61TXREF 




J61 Q1I2 TIME FOR REFERENCE SKILLS 


10,512 


78.7 


J61TXART 




J61 Q1G2 TIME FOR ART 


9,563 


80.6 



See note at end of table. 
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Table 6-20. Items on the ECLS-K fifth-grade restricted-use data file with less than 85 percent 
response rates: School year 2003-04 — Continued 

Number Item response rate 



Variable name 


Description 


eligible 


(percent) 


Teacher-level questionnaire (continued) 






J61TSTPR 


J61 Q19 HOURS SPENT IN STD TEST PREP 


9,688 


81.1 


J61TXMUS 


J61 Q1F2 TIME FOR MUSIC 


8,940 


82.6 


Child-level mathematics teacher questionnaire 






M6LNGOS 


M6 Q17D LANGUAGE OF MATHEMATICS INSTRUCTION 


6 


83.3 


Special education teacher questionnaire B 






E6EVLOTH 


E6 Q22H OTHER EVALUATION 


981 


76.6 


E6DKMTHD 


E6 Q14M DON’T KNOW METHODS USED 


981 


81.2 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten 
Class of 1998-99 (ECLS-K), spring 2004. 
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7. WEIGHTING AND VARIANCE ESTIMATION 



The ECLS-K data were weighted to compensate for differential probabilities of selection at 
each sampling stage and to adjust for the effects of nonresponse. In the ECLS-K base year, weights were 
computed at the child, school and teacher levels. Estimates using the base year weights are representative 
of all kindergarten children, all schools with kindergarten programs and all kindergarten teachers. After 
the base year, only child-level weights were computed. The use of these weights was essential to produce 
estimates that are representative of the cohort of children who were in kindergarten in 1 998-99 or in first 
grade in 1999-2000. In first grade, the sample was freshened with first-graders who had not been sampled 
in kindergarten or first grade. Freshening was not done in third grade or in fifth grade. Estimates from the 
ECLS-K fifth-grade data are representative of the population cohort rather than of all fifth-graders in 
2003-04. While the vast majority of children in fifth grade in the 2003-04 school year are members of the 
cohort, fifth-graders who repeated fourth or fifth grade, children who were homeschooled before enrolling 
in fifth grade, and recent immigrants are not covered. Data were collected from teachers and schools to 
provide important contextual information about the school environment for the sampled children. 
Similarly, home environment data were collected from parents. Data from these sources are not 
representative of all fifth-grade parents, teachers, and schools in 2003-04. For this reason, the only 
weights produced from the study are child-level weights for making statements about children, including 
statements about the parents, teachers, and schools of those children. 

The different types of weights are discussed in section 7.1, followed by a detailed 
description of the computation of the weights in section 7.2. Section 7.3 describes the variance estimation 
methods suitable for the ECLS-K. 



7.1 Types of Weights 

Two sets of weights were computed for fifth grade, cross-sectional and longitudinal. The use 
of these weights is essential to produce estimates that are representative of the cohort of children who 
were in kindergarten in 1998-99 or in first grade in 1999-2000. Since the ECLS-K sample was not 
freshened after the first-grade year with third- or fifth-graders who did not have a chance to be sampled in 
kindergarten or first grade (as was done in first grade), estimates from the ECLS-K third- and fifth-grade 
data are representative of the population cohort rather than all third-graders in 2001-02 or all fifth-graders 
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in 2003-04. While the vast majority of children in third grade in the 2001-02 school year and in fifth 
grade in the 2003-04 school year are members of the cohort, third-graders who repeated second or third 
grade, fifth-graders who repeated third or fourth grade, and recent immigrants are not covered. The fifth 
grade cross-sectional weights are used for analyses of data from the fifth-grade data collection round. The 
fifth grade longitudinal weights are used for analyses of data from a longitudinal file including fifth-grade 
data in conjunction with data from one or more other rounds. 

As in previous years, there are several survey instruments administered to sampled children 
and their parents, teachers and schools: cognitive and physical assessments for children; self-description 
child questionnaire (third and fifth grade only), parent instruments; several types of teacher instruments 
completed by reading, mathematics, science and special education teachers; and school instruments. The 
stages of base year sampling in conjunction with differential nonresponse at each stage and the diversity 
of survey instruments require that multiple fifth-grade cross-sectional sampling weights be computed for 
use in analyzing the fifth-grade ECLS-K data. Several combinations of kindergarten through fifth-grade 
longitudinal weights were also computed. Exhibit 7-1 summarizes the different types of cross-sectional 
weights. 
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Exhibit 7-1. ECLS-K fifth-grade cross-sectional weights: School year 2003-04 



Weight 


To be used for analysis of... 


C6CW0 


Fifth-grade direct child assessment data, alone or in conjunction with any combination of 
(a) a limited set of child characteristics (e.g., age, sex, race/ethnicity), (b) teacher-level 
data from any fifth-grade teacher questionnaire without child-level teacher data, or (c) 
data from the school administrator questionnaire or school facilities checklist. 


C6PW0 


Fifth-grade parent interview data alone or in combination with (a) fifth-grade child 
assessment data, (b) data from any fifth-grade teacher questionnaire (teacher- level or 
child-level), or (c) data from the school administrator questionnaire or school facilities 
checklist. 

Exception : If data from the parent AND child assessment AND teacher (child- and/or 
teacher-level) are used together, then either C6CPTR0, C6CPTM0 or C6CPTS0 should 
be used. 


C6CPTR0 


Fifth-grade direct child assessment data combined with fifth-grade parent interview data 
AND fifth-grade teacher-level data with or without child-level data from the reading 
teacher, alone or in conjunction with data from the school administrator or facilities 
checklist. 


C6CPTM0 


Fifth-grade direct child assessment data combined with fifth-grade parent interview data 
AND fifth-grade child data from mathematics teacher (with or without teacher-level 
data), alone or in conjunction with data from the school administrator or facilities 
checklist. 


C6CPTS0 


Fifth-grade direct child assessment data combined with fifth-grade parent interview data 
AND fifth-grade child data from science teacher (with or without teacher-level data), 
alone or in conjunction with data from the school administrator or facilities checklist. 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



The ECLS-K longitudinal file is created by merging data from the base year, first grade, 
third grade, and fifth grade. Longitudinal weights were created to use in analyzing data in this 
longitudinal file. These weights are described in exhibit 7-2. All longitudinal weights are child-level 
weights. 
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Exhibit 7-2. ECLS-K: K-5 longitudinal weights, spring- fifth grade: School year 2003-04 



Weight 

C56CW0 

C56PW0 

C456CW0 

C456PW0 

C26FC0 

C26FP0 

C16FC0 

C16FP0 

C16SC0 

Cl 6SP0 



To be used for analysis of... 

child direct assessment data from BOTH spring-third grade and spring-fifth grade, alone 
or in conjunction with any of the school or teacher data, or a limited set of child 
characteristics (e.g., age, sex, and race/ethnicity). 

parent interview data from BOTH spring-third grade and spring-fifth grade, alone or in 
conjunction with any of the child assessment, school, or teacher data. 

child direct assessment data from spring-first grade AND spring-third grade AND spring- 
fifth grade, alone or in conjunction with any of the school or teacher data, or a limited set 
of child characteristics (e.g., age, sex, and race/ethnicity). 

parent interview data from spring-first grade AND spring-third grade AND spring-fifth 
grade, alone or in conjunction with any of the child assessment, school, or teacher data. 

child direct assessment data from FOUR rounds of data collection involving the FULL 
sample of children (spring-kindergarten, spring-first grade, spring-third grade, and 
spring-fifth grade), alone or in conjunction with any of the school or teacher data, or a 
limited set of child characteristics (e.g., age, sex, and race/ethnicity). 

parent interview data from FOUR rounds of data collection involving the FULL sample 
of children (spring-kindergarten, spring-first grade, spring-third grade, and spring-fifth 
grade), alone or in conjunction with any of the child assessment, school, or teacher data. 

child direct assessment data from FIVE rounds of data collections involving the FULL 
sample of children (fall-kindergarten, spring-kindergarten, spring-first grade, spring-third 
grade, and spring-fifth grade), alone or in conjunction with any of the school or teacher 
data, or a limited set of child characteristics (e.g., age, sex, and race/ethnicity). 

parent interview data from FIVE rounds of data collections involving the FULL sample 
of children (fall-kindergarten, spring-kindergarten, spring-first grade, spring-third grade, 
and spring-fifth grade) alone or in conjunction with any of the child assessment, school, 
or teacher data. 

child direct assessment data from ALL SIX rounds of data collection (fall-kindergarten, 
spring-kindergarten, fall-first grade, spring-first grade, spring-third grade, and spring-fifth 
grade), alone or in conjunction with any of the school or teacher data, or a limited set of 
child characteristics (e.g., age, sex, and race/ethnicity). 

parent interview data from ALL SIX rounds of data collection (fall-kindergarten, spring- 
kindergarten, fall-first grade, spring-first grade, spring-third grade, and spring-fifth 
grade), alone or in conjunction with any of the child assessment, school, or teacher data. 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



As mentioned in the introduction, teachers and schools are not representative of fifth-grade 
teachers and schools in 2003-04. For this reason, there are no cross-sectional weights computed to 
provide estimates at the school or teacher level. Consequently, there are no longitudinal weights 
computed at the school or teacher level. 
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Each set of weights created to be used with the ECLS-K data consists of a full sample weight 
that is used in computing survey estimates and replicate weights that are used in variance estimation with 
a jackknife replication method. First-stage stratum and primary sampling unit (PSU) identifiers are also 
created so that variance estimation using the Taylor series approximation method can be produced using 
the full sample weights. See section 7.2.6 for a description of how replicate weights were created. Section 
7.3 discusses variance estimation methods. 

The data file includes the final full sample weight (described in section 7.2.4) and the final 
replicate weights (described in section 7.2.6) but not the intermediate weights leading to the final weights. 
The names of the full sample weights in the file are as described in exhibits 7-1 and 7-2 (e.g., C6CW0). 
The names of the replicate weights have the same prefix as the full sample weight with the last digit 
indicating the replicate (e.g., C6CW1 to C6CW90 are the 90 replicate weights to be used with the full 
sample weight C6CW0). 



7.2 Computation of the Fifth-Grade Weights 

Among the 21,357 children who were eligible for the study after the base year (21,292 base 
year respondents and 165 children sampled in first grade), the fifth-grade sample excluded 5,214 children 
as explained in section 3.6.1. As in third grade, only a subsample of children who moved from the schools 
they were attending when they were sampled originally was followed into their new schools. Elowever, 
children who moved into a destination school 1 because they had completed the highest grade at the 
originally sampled school were all followed. The fifth-grade subsampling of movers continues to give 
more weight to children in the language minority group (i.e., movers in this group were subsampled at 
higher rates than non-language minority movers). Other smaller groups of movers were also subsampled 
at lower rates, such as selected groups of movers who were sampled in first grade (as compared with base 
year respondent movers), and movers who did not have full longitudinal data. Differential sampling rates 
of movers are presented in section 3.6.1. Another feature of the fifth-grade sample is the subsampling of 
children for the administration of the mathematics or science questionnaires as discussed in section 3.6. 

In the weighting procedures, children excluded from the fifth-grade data collection are 
considered ineligible if they became ineligible in an earlier round (because they had died or moved out of 
the country), as movers not subsampled for follow-up if they were subsampled out in previous rounds 

1 A destination school is a school that received at least four students from the school where they had just completed the highest grade. 
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because they moved out of the original sample, or of u nk nown eligibility if they were hard refusal cases 
or if they had neither first-grade nor third-grade data. Excluded children are properly adjusted for in the 
weighting procedures. 



The weighting procedures for both cross-sectional and longitudinal weights are similar, 
although weighting cells were defined differently for each type of weight. For example, any longitudinal 
weight that contains data from fall-first grade may have used different cells due to sample size 
constraints. 



The weighting procedures for the fifth grade were divided into three main stages. These 
procedures were followed for creating each weight shown in exhibit 7-1 and exhibit 7-2. The change in 
the procedures pertains only to the change in the eligibility of children for whom the weight applies. For 
example, weight C6CW0 pertains to children with completed assessments in fifth grade; weight C56PW0 
pertains to children with completed parent interview in both third grade and fifth grade. In the base year, 
children who were not assessed because of a disability or because they were language minority children 
had positive C1CW0 and C2CW0 weights because they had data such as age, sex, race/ethnicity, height 
and weight, and characteristics of parents, teachers, and classrooms. In subsequent rounds of data 
collection, they continued to be treated the same. 

Weights that include any fall-first grade data (such as C16SC0, which is the weight for 
children for whom child assessments were obtained in all six rounds) were computed using the same 
procedures, but the cells for the weighting adjustments were more restricted because only the fall-first 
grade subsample was included. The replication scheme for data that include the fall-first grade panel is 
also different as described in section 7.2.6. 

The first stage of weighting was to compute an initial child weight that reflects the 

following: 



■ Adjustment of the school base weight for base year school-level nonresponse; 

■ Adjustment of the child weights for base year child-level nonresponse; and 

■ Adjustment of the base year child weight for subsampling of schools for freshening in 
first grade (for children sampled in first-grade only). 
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The procedures used in this first stage are the same as in all rounds of data collection after 
the base year because the same sample of children (base year respondents and children sampled in first 
grade) is eligible for subsequent rounds of data collection. 

The second stage of weighting was to adjust the initial child weight computed in the first 
stage for the following: 

■ Subsampling of movers; and 

■ Child-level nonresponse. 

For the mathematics and science child-parent-teacher weights, an additional adjustment was 
necessary (before the second stage adjustment for the subsampling of movers and for nonresponse) to 
adjust for the subsampling of children for whom mathematics or science teacher data questionnaires were 
administered. This adjustment is described in section 7.2.5. 

The third and last stage was to rake the weights adjusted in the second stage to sample-based 
control totals. Raking is a multivariate poststratification of the weights, explained in section 7.2.4. 

In general, in each adjustment to the weight, the adjustment factor is multiplied by the 
weight in the prior step to get the adjusted weight. This fact is not repeated in the discussions of the 
weight adjustments in the following sections; only the computation of the adjustment factor is discussed. 



7.2.1 Initial Child Weights 

As mentioned earlier, the first stage of weighting was to compute an initial child weight that 
reflects: (1) the adjustment of the school base weight for base year school-level nonresponse (school-level 
weights), (2) the adjustment of the child weights for base year child-level nonresponse (child-level 
weights), and (3) the adjustment of the base year child weight for subsampling of schools for freshening 
in first grade (child-level weights, for children sampled in first grade only). These weights were already 
computed for spring-first grade. For completeness, they are described below, in section 7.2. 1.1 for the 
school-level weights, and in section 7.2. 1.2 for the child-level weights. 
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7.2.1. 1 



Base Year Nonresponse-Adjusted School Weights 



The school base weight^ SCHLBW i was the same as that computed for previous rounds of 
data collection. It is the inverse of the probability of selection of the school from a stratified probability 
proportional to size (PPS) design: 

SCHLB W ; = x , 

P° s PSu j P0S SCHLi 

where 

POS PSUj is the probability of selection of the PSU j, and 
POS SCHL , is the probability of selection of school i within the PSU j, 

where the probability of selection of the PSUy, POS FSUj , was defined as 

1 if PSU j is a self-representing (SR) PSU, 

POSpsu i=\( 2 MA 

J — if PSU j is a non self-representing (NSR) PSU, 

M h J J 

where 

M . is the measure of size (MOS) of PSU (i.e., count of five -year-old children in the PSU as 
described in section 3.2.1), and 
M h is the total MOS in stratum h, PSU . e h , 

and the probability of selection of school i within PSU j , POS SCHL ,■ , was defined as 

m. 

POS O/— r TTT • = M 1 • ^ , 

SLHLi kj ^ m _ 

iekj 

where 

n kj is the target number of schools in stratum k in PSU i, and 
m j is the MOS of the school i in stratum k in PSU i. 

For schools sampled using the new school sampling procedure, 3 the school base weight 
SCHLB Wj was computed as 

2 Only schools sampled in the base year have base weights. Transfer schools do not have base weights, but children in transfer schools carry with 
them the base weights of the original sampled schools. 
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SCHLBWf = 



1 



POS 



PSUj 



J_ 1 

/. POS SCHLi 



and the factor f t is defined as 



fi 



1 

PLEA 
VP DIO 



if the school is a new non-Catholic private school, 
if the school is a new public school, 
if the school is a new Catholic school, 



where 

P LEA ' s the within stratum selection probability of the school district, and 
p DI0 is the within stratum selection probability of the diocese. 



The school base weight was adjusted for base year school-level nonresponse. A base year 
responding school was an original sample school with at least one child with a positive C1CW0, C2CW0, 
C1PW0, or C2PW0 weight. C1CW0 is positive for language minority (not Spanish) children who were 
screened for English proficiency (regardless of whether they went on to take the assessments), children 
with disabilities, and children with at least one direct cognitive test score in fall-kindergarten. C1PW0 is 
positive for children whose parents completed the family structure questions of the parent interview in 
fall-kindergarten. C2CW0 and C2PW0 weights are positive under similar circumstances but for spring- 
kindergarten. Schools that did not meet this condition are nonrespondents and their weights were 
distributed across responding units (at the school level) in this stage. The base year school weight was 

adjusted within nonresponse weighting classes. The base year nonresponse-adjusted school weight 
SCHLADWj was computed as 



SCHLADW ] = SCHLADF r x SCHLBW ,, 

L '-'l 

where 

SCHLB W t is the school base weight, and 

SCHADF c is the base year school nonresponse adjustment factor for schools in cell c, calculated 
as 



SCHLADF i = 



X SCHLBW. x m. 

i c PR ,ENR 

c c 

x SCHLBW. xm. 

i i 

i e ER 

c 



The sample was expanded to account for newly opened schools and schools not found in the sampling frame. See chapter 3 for more 
details on how these schools were identified. 
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where 



ER C denotes the set of eligible school respondents in cell c, 

ENR C denotes the set of eligible school nonrespondents in cell c, and 

777 , is the measure of size for school i (i.e., count of students in the school as described in section 
3.22.2) 

Nonresponse cells were created using the Chi-squared Automatic Interaction Detector 
(CHAID) and variables with known values for both respondents and nonrespondents. Base year school 
characteristics used for constructing nonresponse cells were the school affiliation (public, Catholic 
private, non-Catholic religious private, or nonreligious private), the school locale (large city, midsize city, 
suburb of large city, suburb of midsize city, large town, small town, or rural area), the region where the 
school is located (Northeast, Midwest, South, or West), and the size classification of the school in terms 
of school enrollment as described in table 7-1. 

Table 7-1. Size classification for school nonresponse adjustment: 

School year 1998-99 



Size 

classification 


School enrollment 




Public 


Private 


1 


1-24 


1 - 11 


2 


25-39 


12-23 


3 


40-49 


24-35 


4 


50-59 


36-47 


5 


60-69 


48-59 


6 


70-79 


60 or more 


7 


80-89 


t 


8 


90-99 


t 


9 


100-119 


t 


10 


120-139 


t 


11 


140-179 


t 


12 


180-219 


t 


13 


220 or more 


t 



(• Not applicable. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early 
Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), spring 1999. 
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7.2. 1.2 



Base Year Child Weights 



Two groups of children were fielded in fifth grade: base year respondents, and eligible 
children who were sampled in first grade as part of the sample freshening procedure. The base year child 
weights for the two groups were the same as those computed for the first-grade year. A description of the 
two weights follows. 



7.2. 1.2.1 Base Year Child Weights for Base Year Respondents 

A base year respondent is a sampled child with a positive C1CW0, C2CW0, C1PW0, or 
C2PW0 weight. The C1CW0 weights are positive for children belonging in the language minority (not 
Spanish) group who were screened for English proficiency (regardless of whether they went on to take the 
assessments), children assessed in fall-kindergarten, and children excluded from assessment because of a 
disability. The C1PW0 weights are positive for children whose parents completed the family structure 
questions of the parent interview in fall-kindergarten. The C2CW0 and C2PW0 weights are positive 
under similar circumstances, but apply to data from spring-kindergarten. 

The base year child weight BYCHLDW- was computed as 

BYCHLDW ,■ = SCHLADW: x (1 / CHLD _ PROB . ), 

ll l 

where 

SCHLADW • is the base year nonresponse-adjusted school weight described in section 7.2. 1.1, 
CHLD PROB , is the probability of selection of the child within a school. 

To account for base year nonresponse — children who were not assessed in the base year and 
whose parent interviews were not completed (i.e., children who did not have at least one positive weight 
among C1CW0, C2CW0, C1PW0 and C2PW0) — the base year child weight was adjusted for 
nonresponse. The child weight adjusted for base year child-level nonresponse, ABYCHLDW-, was 
computed as 



ABYCHLDW: = BY ADF r x BYCHLDW: , 

i c L 

where BYADF C , the adjustment factor, was calculated as 
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x BYCHLDW : 

BYADF C - , 

c x BYCHLDW- 

ieBY_R c 

where 

BY_R C denotes the set of base year child respondents in cell c, and 
BY_NR c denotes the set of base year child nonrespondents in cell c. 

The base year child weights were adjusted using weighting classes similar to those 
developed for the cross-sectional spring-kindergarten child weights. These classes were created with 
CHAID, using the school characteristics from the school nonresponse adjustments (i.e., school affiliation, 
locale, region, school enrollment classified into size category), and a set of child characteristics (i.e., year 
of birth, sex, and race/ethnicity). Data on year of birth were obtained from the parent interviews, while 
data on sex and race/ethnicity were from the child sampling information, which was provided by the 
schools. If year of birth was missing from the parent interview, then it was taken from the child sampling 
information. If sex or race/ethnicity was missing from the child sampling information, then they were 
obtained from the parent interview data. Any remaining missing data were imputed with the modal value 
from the school from which the child was sampled for this purpose. 



7.2. 1.2.2 Base Year Child Weights for Children Sampled in First Grade 

In spring-first grade the student sample was freshened to include first-graders who had not 
been enrolled in kindergarten in 1998-99 and, therefore, had no chance of being included in the ECLS-K 
base year kindergarten sample. For this group of children who entered the sample in first grade, their 
weights need to have additional adjustments to account for the freshening procedure. See chapter 3 for a 
discussion of the student freshening in spring-first grade. 

Since each child sampled in first grade was directly linked to a child sampled in 
kindergarten, the first step was to compute a weight for the children who were sampled in kindergarten 
that reflects the school freshening subsampling and the school freshening nonresponse (some schools 
refused to provide the complete alphabetical roster of all students enrolled in first grade needed for 
freshening). This weight was then linked back to the child sampled in first grade and further adjusted for 
nonresponse because the data (e.g., assessment data, parent interview data) had not been obtained from 
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the sample of freshened children. The procedures for computing the base year child weights for children 
sampled in first grade are described next. 

School Weight Adjusted for Subsampling of Schools for Freshening. The school base 
weight adjusted for base year school- level nonresponse ( SCHLADW ■ ) computed in section 7. 2. 1.1 was 
adjusted for the subsampling of schools for freshening. As noted earlier, student freshening was done in a 
50 percent subsample of schools. The school weight adjusted for school freshening subsampling, 
FRXSCHW j , was calculated as 



FRXSCHW ; = FR\ADF r x SCHLADW:, 

l c L 



where FR\ADF C , the adjustment factor, was computed as 



FR\ADF C 



X _ SCHLADW i 
i<=F c uF c 

X SCHLADW: 
iGF c 

0 



if school/ e F , 
c 

if school/ g F , 
c 



where 

F c denotes the set of schools subsampled for freshening, and 
F ' denotes the set of schools not subsampled for freshening. 



This adjustment was done within cells defined by school affiliation (public, Catholic private, 
non-Catholic religious private, or nonreligious private) and census region (Northeast, Midwest, South, or 
West). Adjustment cells were created using CHA1D. 



School Weight Adjusted for Freshening Nonresponse. The freshening procedure could 
not be applied in all designated schools because some schools did not provide the information needed for 
freshening (see chapter 3 for more details on the freshening procedures). These schools were considered 
nonrespondents. The school weight adjusted for freshening school-level nonresponse, FRISCHWj , was 
computed as 



FR2SCHW '■ = FR2ADF r x FRXSCHW- , 

l c l 
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where FR2ADF C , the adjustment factor, was computed as 



X FRISCHW. x m . 

i e FER u FENR 1 1 

FR2ADF '■ = £ £ , 

1 X FRISCHW. Km. 

i i 

i e FER 

c 

where 

m- is the original school MOS, 

FER denotes the set of freshening school respondents in cell c, and 
FENR C denotes the set of freshening school nonrespondents in cell c. 

In both the numerator and denominator of this factor, the school measure of size (i.e., the 
count of students in the school as described in section 3. 2. 2.2) was incoiporated; the school measure of 
size is relevant because the weights will be used for child-level estimates, not school-level estimates. The 
nonresponse cells for this adjustment were created with CHA1D using school affiliation (public, Catholic 
private, non-Catholic religious private, or nonreligious private) and type of locale (large city, midsize city, 
suburb of large city, suburb of midsize city, large town, small town, or rural area). 

Base Year Child Weight. Next, the school-adjusted weight was multiplied by the inverse of 
the probability of sampling the child in the base year to obtain a base year child weight for freshening. 
The base year child weight was B YCHLD W t : 

BYCHLDW i = FR2SCHW t x (\/CHLD PROB.), 
where CHLD PROB- is the within-school child selection probability. 

The base year child weight was then adjusted for base year child nonresponse because 
children who did not respond in the base year could not be linked to children in first grade in spring 2000. 
The adjusted weight, A B YCHLD W t was computed as 

AB YCHLD W: = BYADF r x B YCHLD W ; , 

l c i 
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where BYADF C , the adjustment factor, was calculated as 



Z BYCHLDW : 

BYADF C - , 

6 z BYCHLDW- 

i&BY_R c 1 

where 

BY_R C denotes the set of base year child respondents in cell c, and 
BY_NR c denotes the set of base year child nonrespondents in cell c. 

The nonresponse cells were created with CHAID using the school characteristics school 
affiliation, locale, region, and school enrollment size, and the child characteristics age, sex, and 
race/ethnicity. 

Base Year Child Weights Adjusted for Movers. Only children who did not move from 
their original school were designated as links to children in the freshening procedure. The children who 
moved and were followed into their new schools were not identified to participate in the freshening 
process in their new schools. As a result, all children who moved were considered nonrespondents for the 
freshening process. Additionally, nonmovers and movers who were not in first grade were not eligible for 
freshening (e.g., if the child was in kindergarten in spring 2000, he or she would be linked only to other 
kindergarten children and thus was not eligible for the freshening of first-graders). An adjustment was 
necessary to account for these two groups of children and was done in two steps. 

In the first step, an adjustment was made for movers whose grade was unknown. A portion 
of the movers was assumed to be in first grade. In the second step, the weights were adjusted for children 
who were in first grade and who were not identified to participate in the freshening process because they 
moved into a new school. For this two-step adjustment, each child was classified as in table 7-2. 
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Table 7-2. Groups of children defined for mover adjustments: 
School year 1999-2000 



Groups 


Mover status 


MO VI 


Mover enrolled in first grade 


MOVOTH 


Mover enrolled in another grade 


MUNK 


Mover enrolled in an unknown grade 


NMl 


Nonmover enrolled in first grade 


NMOTH 


Nonmover enrolled in another grade 



SOURCE: U.S. Department of Education, National Center for Education 
Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 
(ECLS-K), spring 2000. 



The first step adjustment for movers whose grade in spring 2000 was unknown was 

computed as 



R4MOVW ] = R4MOVFI, x ABYCHLDW:, 

l c L 



where R4MOVF\ c , the adjustment factor, was computed as 



R4MOVF\ c = 



iGMOVl c vMOVOTH c uMUNK c 



ABYCHLDW i 



ABYCHLDW i 



0 



if child g MOVIvj MOVOTH incellc, 
if child g MUNK incellc, 



1 1 if child g NM\ u NMOTH in cellc. 

The second step adjustment for movers who could not be used as links for freshening was 



computed as 



R4MOVW2 ■ = R4MOVF2xR4MOVFY, 

l c l 

where R4MOVF2 c , the adjustment factor, was computed as 



R4MOVF 2 C 



X R4MOVWV 

mNMl c uMOVl c 

x R4MOVWY 

ieNMl c 

0 



1 



if child g NM 1 incellc, 

if child g MOV 1 incellc, 

if child g NMOTH u MO VOTH in cellc. 
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This two-step adjustment was done within cells defined by school affiliation and census 



region. 



The weights thus created for children sampled in kindergarten were then linked to the 
children that they brought into the sample in first grade through sample freshening. In other words, the 
weight of the child sampled in first grade was defined at this point to be the weight computed for the child 
sampled in kindergarten that was responsible for bringing the first-grader into the sample. 

For the next step in the computation of the fifth-grade child weights, the two groups of 
children — base year respondents and children sampled in first grade through sample freshening — were 
put together, and a common variable and label were used to designate the initial child weight. This is the 
base year child weight as computed above for each group of children: 

f ABYCHLDW ] if base year respondent, 

ICHLDW: - \ 

I R4M()VW2 I if sampledinfirstgrade. 

The initial child weights /CE/LDfrj were adjusted for movers between the base year and fifth 
grade and for nonresponse in fifth grade, and raked to sampled-based control totals to obtain the final 
fifth-grade child weights. These adjustments and raking procedures are described below. 



7.2.2 Adjustment for Movers Between the Base Year and Fifth Grade 

First, the initial child weights were adjusted to reflect the subsampling of movers. In the 
ECLS-K, a child could move more than once and at different times. For example, a child could move out 
of his original sample school because the school did not have grades higher than kindergarten. Then he 
could move again between first and third grade, first and fifth grade, or third and fifth grade. Once a child 
was identified as a mover, he stayed a mover unless he moved back to the original sample school. For 
example, a child who moved between kindergarten and third grade, but stayed in that same school 
between third and fifth grade, was considered a mover for the fifth grade. 

Each mover in the fifth grade had a flag indicating whether he was followed into the new 
school. These flags were set according to the mover subsampling plan described in section 3.6.1. Children 
who were excluded from the fifth-grade data collection because they had moved out of the original 
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schools and were subsampled out for followup in previous rounds had their flag set to “not followed.” In 
fifth grade, children were fielded as described in exhibit 7-3. 



Exhibit 7-3. Movers and nonmovers by retention status: School year 2003-04 



Child moved out of original school 


Child subsampled for followup 


Child fielded 
in fifth grade 


Before 
fifth grade 


During 
fifth grade 


Before 
fifth grade 


During 
fifth grade 


No 


No 


t 


t 


Yes 


No 


Yes 


t 


No 


No 


No 


Yes 


t 


Yes 


Yes 


Yes 


No, did not move again 


No 


t 


No 


Yes 


No, did not move again 


Yes 


No 


No 


Yes 


No, did not move again 


Yes 


Yes 


Yes 


Yes 


Back in original school 


t 


t 


Yes 



fNot applicable. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), fall 1998, spring 1999, fall 1999, spring 2000, spring 2002, and spring 2004. 



The initial child weight described in section 7.2.1 was adjusted to reflect this subsampling 
movers. The initial child weight adjusted for movers R6CHILDWI- was computed as 



R6CHLDWI: = R6ADF\ r x ICHLDW; , 

l ^ l 



where R6ADFI- , the adjustment factor, was computed as 



R6ADF\ C 



1 

X ICHLDW ) 

ieMOVER with 
FOLLOW C _F in (0,1) 

X ICHLDW: 

UgMOVER with 
FOLLOW V=1 

0 



if child in cell c is not a mover, 



if childincell c isamoverand FOLLOWF - 1, 



if childincell c isamoverand FOLLOWF = 0, 



and MOVER denotes the set of children that are movers in cell c. 
c 



For the cross-sectional weights, the mover adjustment factor was computed within cells 
created with CHAID using the following characteristics: whether children were sampled in kindergarten 
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or first grade, and whether they were language minority children. 4 For the longitudinal weights, a 
longitudinal mover follow status was created that takes into account whether the child moved from his 
original school in fall-first grade, spring-first grade, or spring-third grade (for longitudinal weights 
involving the fall-first grade data) or whether the child moved from his or her original school in spring- 
first grade or spring-third grade (for the other longitudinal weights). If a child moved in either round, he 
was considered a mover. Appendix A gives the cell definitions for the mover adjustment for cross- 
sectional and longitudinal weights. 

Twelve children with large weights had their weights trimmed by 40 percent. The trimming 
procedure was the same as in previous years. However, the weights were not redistributed because the 
total sum of weights was re-established in the raking procedure that came later. 



7.2.3 Adjustment for Fifth-Grade Nonresponse 



After the adjustment for subsampling movers, the child weights were adjusted for 
nonresponse. As in spring-first grade and spring-third grade, the nonresponse adjustment was done in two 
steps. In the first step, the adjustment was for children whose eligibility was not determined (unknown 
eligibility). A portion of children of unknown eligibility was assumed to be ineligible, equal to the 
proportion of children of known eligibility who are ineligible. In the second step, the adjustment was for 
eligible nonrespondents. To carry out these adjustments, each child was classified as (a) an eligible 
respondent, (b) an eligible nonrespondent, (c) ineligible (out of the country or deceased) or (d) of 
unknown eligibility (mover who could not be located), as shown in table 7-3. 



Table 7-3. Groups of children defined for nonresponse adjustments: 
School year 2003-04 



Groups 


Eligibility and response status 


ER 


Eligible respondent 


ENR 


Eligible nonrespondent 


IN 


Ineligible (out of the country or deceased) 


UNK 


Unknown eligibility (mover who could not be located) 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early 
Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 



4 Fewer characteristics were used than in previous years to create cells for mover adjustments. This is due to cells with a small number of records, 
requiring them to be collapsed in order to avoid large adjustment factors. This resulted in fewer cells, hence fewer characteristics being used. 
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The child weight adjusted for nonrespondents with unknown eligibility, R6CHLDW2 j, was 



computed as 



R6CHLDW2 . = R6ADF2 r x R6CHLDW h, 

l L 



where R6ADF2 C , the adjustment factor, was computed as 



R6ADF2 C 



X R6CHLDW 1, 

i&ER c vjENR c vJlN c vjUNK c 

x R6CHLDWI- 

ieERc^iENR c uIN c 

0 



if child e £7? u ENR u //V in cellc, 
if child e UNK in cellc. 



In the second adjustment, the child weight was adjusted for eligible nonrespondents. The 
child weight adjusted for eligible nonrespondents, R6CHLDW3-, was computed as 

R6CHLDW3 i = R6ADF3 C x R6CHLDW2 i , 



where R6ADF3 C , the adjustment factor, was computed as: 



R6ADF3 C 



X R6CHLDW2 , 

i£ER c uENR c 

x R6CHLDW2 ■ 

0 



1 



if child e ER in cellc, 
if child e ENR in cellc, 
if child e IN in cellc. 



In both steps of the adjustment, separate nonresponse classes were created for movers and 
nonmovers with CHAID using various combinations of response status of child assessments and parent 
interviews in the base year as well as whether children belong to the language minority group, the type of 
household collected from the parent interviews (all cross-sectional weights except C6CW0), and the 
school affiliation including whether the child was homeschooled (C6CPTR0, C6CPTM0 and C6CPTS0 
only). The adjustment cells for each type of weight are shown in appendix B. Where applicable, very 
large nonresponse adjusted weights were trimmed by 40 percent. As before, the weights were not 
redistributed in each case because the total sum of weights was re-established in the raking step that came 
next. 
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7.2.4 



Raking to Sample-Based Control Totals 



To reduce the variability due to the subsampling of schools and movers, the child weights 
were then raked (i.e., calibrated) to sample-based control totals computed using the initial child weights 
described in section 7.2.1. A file was created with the initial child weights and school and child 
characteristics collected in the base year or first-grade year (such as school affiliation, region, urbanicity, 
sex, age, race/ethnicity, SES, language minority status, whether sampled in kindergarten or first grade, 
and if sampled in kindergarten, mover status in spring-first grade) to be used in the computation of the 
control totals. The child records included in this file are records of base-year respondents and records of 
eligible children sampled in first grade, including records of children who became ineligible in fifth 
grade. The sum of the initial weights thus calculated is the estimated number of children who were in 
kindergarten in 1998-99 or first grade in 1999-2000. In the previous steps, the weights of the 
nonresponding children were distributed to the responding children while the weights of the ineligible 
children were not affected by this weighting step. The weights of the ineligible children were set to zero 
at the end of this process because these children are not meant to be included in the analysis of the fifth- 
grade data. The reason for including the ineligible children in the raking step is that these children were 
included in the sampled-based control totals. 

The raked child weight or fifth-grade final child weight, R6CHLDW4 - , was calculated as 



R6CHLDW4 i = R6ADF4 C x R6CHLDW3 i , 



where R6ADF4 C , the raking factor for raking cell c, was computed as 



R6ADF4= 



SMP _CNT c 
Y J R6CHLDW3 i ’ 

iec 



where SMP _CNT c is the sample-based control total for raking cell c. Weights of children who become 
ineligible in fifth grade were set to zero after this step. 



This raking procedure is essentially a multivariate poststratification. Raking cells (also 
known as raking dimensions because they typically involve more than one variable, for example, sex by 
age) were created with CHAID using school and child characteristics collected in the base year or first- 
grade year: school affiliation, region, type of locale, sex, age, race/ethnicity, socio-economic scale (SES), 
language minority status, whether sampled in kindergarten or first grade, and if sampled in kindergarten, 
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mover status. These characteristics come from the fifth-grade data collection or previous rounds of data 
collection if fifth-grade data are missing. Appendix C gives the raking dimensions used for fifth grade. 

There was no restriction set in the number of iterations during the raking procedure. The 
procedure was allowed to run until complete convergence was achieved within a control total. This 
occurred after 12 to 19 iterations. 



7.2.5 Additional Adjustment for Child-Parent-Teacher Cross-Sectional Weights 

A new feature of the fifth-grade sample is the subsampling of children for the administration 
of the mathematics and science teacher questionnaires. While all children had child-level questionnaires 
filled out by their reading teachers, half were subsampled to have child-level questionnaires filled out by 
their mathematics teachers and the other half had child-level questionnaires filled out by their science 
teachers. For this reason, there are three child-parent-teacher weights that will be used to analyze direct 
child assessment data combined with parent interview data and child data provided by teachers (in 
conjunction with school level or teacher level data). In all three weights, the presence of at least one 
completed teacher-level questionnaire determines whether a child would have a positive child-parent- 
teacher weight in the two subjects to which they were assigned (i.e., reading and mathematics or reading 
and science.). A child could have one teacher who taught all subjects, in which case the teacher would be 
asked to fill out both the reading questionnaire and the mathematics questionnaire (if the child was 
selected for mathematics) or science questionnaire (if the child was selected for science). A child could 
also have different teachers teaching different subjects, in which case the child may have a reading 
teacher filling out the reading questionnaire and a mathematics teacher filling out the mathematics 
questionnaire, and both teachers could have filled out the teacher-level questionnaire. No children have 
both completed mathematics and science questionnaires because of the subsampling. 

Table 7-4 shows the distribution of children who have direct child assessment data, parent 
interview data and child-level data from the mathematics teacher by the number of teachers they had who 
filled out the teacher-level questionnaire. The first column in this table shows the number of teachers that 
each child had: only one teacher who taught both reading and mathematics, or two teachers, one teaching 
reading and the other teaching mathematics. The second column shows the type of teacher who filled out 
the teacher-level questionnaire. If the child had only one teacher, then it was this teacher — identified in 
the table as the reading teacher — who filled out the teacher-level questionnaire (3,142 cases out of 5,009 
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or 63 percent). If the child had two teachers, then in the majority of cases, both teachers filled out the 
teacher-level questionnaire (1,803 cases out of 5,009 or 36 percent). There are very few cases where only 
one of the two teachers filled out the teacher-level questionnaire. Table 7-5 shows the same information 
for science. Since C6CPTM0 and C6CPTS0 are used for the analysis of child and parent data with data 
from mathematics and science teachers, another option to define these weights is to use the presence of 
child-level data from the mathematics/science teachers. However, tables 7-4 and 7-5 show that by 
considering the presence of teacher-level data in constructing the child-parent-teacher weights, there are 
more records with positive weights for analysis (5,017 cases with nonzero C6CPTM0 compared with 
5,009 in table 7-4; and 5,103 cases with nonzero C6CPTS0 compared with 5,088 in table 7-5). 5 Using 
teacher-level data to define the child-parent-teacher weights is also consistent with previous years’ 
practice. 

Table 7-4. Number of children with direct child assessment, parent interview and child-level data from 
mathematics teacher, by number of teachers who filled out teacher-level questionnaire: 
School year 2003-04 



Number of 
teachers that each 
child had 


Teachers who completed 
teacher-level questionnaire 


Number of children with child-parent-mathematics 
data from the child-level mathematics questionnaire 


Total 




5,009 


1 


Reading 


3,142 


2 


Reading 


25 


2 


Mathematics 


39 


2 


Reading and Mathematics 


1,803 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



5 See section 4.7.7 of the Combined User’s Manual for the ECLS-K Fifth-Grade Data Files and Electronic Codebooks (NCES 2006-032) 
(Tourangeau et al. forthcoming) for a description of the characteristics of the cross-sectional sample weights. 
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Table 7-5. Number of children with direct child assessment, parent interview, and child-level data from 
science teacher, by number of teachers who filled out teacher-level questionnaire: School 
year 2003-04 



Number of 
teachers that each 
child had 


Teachers who completed 
teacher-level questionnaire 


Number of children with child-parent-science data 
from the child-level science questionnaire 


Total 




5,088 


1 


Reading 


2,999 


2 


Reading 


42 


2 


Science 


35 


2 


Reading and Science 


2,012 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



An additional adjustment is necessary to adjust for the subsampling of children for whom 
mathematics or science teacher data questionnaires were administered. 6 Since only half of the children in 
fifth grade were eligible to have a completed mathematics teacher questionnaire and the other half to have 
a completed science teacher questionnaire, the weights before adjustment for movers and nonresponse 
adjustments (described in sections 7.2.2 and 7.2.3, respectively) were adjusted to account for the 
subsampling of children. 



The initial child weight described in section 7.2 was adjusted to reflect the subsampling of 
children for the mathematics teacher questionnaire as 

R6CHLDW0 i = R6ADF0 C x ICHLDW i , 



where R6ADF0 C , the adjustment factor, was computed as 



R6ADF0 = 



X ICHLDW i 
i 

x ICHLDW : ’ 
i&MTH 1 



where MTF[ denotes the set of children subsampled for mathematics teacher questionnaires. 



6 Note that this adjustment occurred prior to the adjustment for subsampling movers, the adjustment for nonresponse, and raking, but it is 
discussed after section 7.2.4 on raking because it only applies to the child-parent-math/science weights. 
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Likewise, the adjustment factor for subsampling children for the science teacher 
questionnaire was computed as 



X ICHLDW i 

R6ADF0 = — , 

X ICHLDW- 
igSCI 

where SCI denotes the set of children subsampled for science teacher questionnaires. 

This adjustment was followed by the adjustments for subsampling movers as discussed in 
section 7.2.2 ( R6CHLDWI ■ with ICHLDW ■ replaced by R6CHLDW0- ), the adjustment for nonresponse 
as discussed in sections 7.2.3 (R6CHLDW2-my4R6CHLDW2>-), and raking as discussed in section 7.2.4 
(R6CHLDW4 i ). 



7.2.6 Replicate Weights 

For each set of cross-sectional and longitudinal weights included in the fifth-grade data file, 
a set of replicate weights was computed. All adjustments to the full sample weights were repeated for the 
replicate weights. The replication scheme used for the base year was used for all of the fifth-grade 
weights that did not contain any fall-first grade component. If a fall-first grade component was included 
in the definition of the respondents for the weight, then the replication scheme used for fall-first grade 
estimates was used. 

Replicate weights are needed to estimate the standard errors of survey estimates. A total of 
90 replicate weights were computed using the paired jackknife method (denoted as JK2) for the fifth- 
grade weights if no fall- first grade component was included. These replicates take into account the Durbin 
method of PSU selection (Durbin 1967). A total of 40 replicates using the paired jackknife method were 
created for the weights that contain a fall-first grade component. The number of replicates is smaller 
because only 30 percent of the full sample of schools was included in the fall-first grade subsample. Only 
one of the two sampled PSUs in the non-self-representing strata was kept in the sample. Consequently, 
the fall-first grade weights do not account for the Durbin PSU sampling method, which required two 
PSUs per stratum. 
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The procedures used to compute the replicate weights took into account each step of the 
weighting process. One feature that is somewhat uncommon in practice is the use of sample-based raking 
as described in section 7.2.4. The control totals (SMP _CNT c ) used for raking are estimates calculated 
using the initial child weights ( ICHLDW - ). When population-based raking is used, these totals are 
assumed to be numbers that are known and without sampling error. To reflect the variability of the control 
totals in the sample-based raking, a set of replicate control totals was calculated rather than having a 
constant set of totals. Each replicate weight was then raked to the corresponding replicate-based control 
total. The result of this process was that each replicate retained the variability associated with the original 
sample estimates of the control totals. As with the full sample weight, the raking procedure was allowed 
to run until complete convergence. For fifth grade, full convergence was achieved after 12 to 19 iterations 
for each replicate weight. 



7.2.6.1 Replicate Weights for Samples Not Involving Fall-First Grade 

For the original ECFS-K design in the base year, replicate weights were created taking into 
account the Durbin method of PSU selection. The Durbin method selects two first-stage units per stratum 
without replacement, with probability proportional to size and a known joint probability of inclusion. 

In the ECFS-K PSU sample design, there were 24 self-representing (SR) strata and 38 non- 
self-representing (NSR) strata. Among the 38 NSR strata, 11 strata were identified as Durbin strata and 
were treated as SR strata for variance estimation. The purpose of the Durbin strata is to allow variances to 
be estimated as if the first-stage units were selected with replacement. This brings the number of SR PSUs 
to 46 (24 original SR PSUs and 22 Durbin PSUs from the 1 1 Durbin strata). The remaining 54 NSR PSUs 
are in 27 NSR strata; thus 27 replicates were formed, each corresponding to one NSR stratum. For the SR 
strata, 63 replicates were formed. The 90 replicates will yield about 76 degrees of freedom for calculating 
confidence intervals for many survey estimates. 

As stated earlier, the sample of PSUs was divided into 90 replicates or variance strata. The 
27 NSR strata formed 27 variance strata of two PSUs each; each PSU formed a variance unit within a 
variance stratum. All schools within an NSR PSU were assigned to the same variance unit and variance 
stratum. Sampled schools in the 46 SR PSUs were grouped into 63 variance strata. In the SR PSUs, 
schools were directly sampled and constituted PSUs. Public schools were sampled from within PSU while 
private schools were pooled into one sampling stratum and selected systematically (except in the SR 
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PSUs identified through the Durbin method where private schools were treated as if they were sampled 
from within PSU). Schools were sorted by sampling stratum, type of school (from the original sample or 
newly selected as part of freshening), type of frame (for new schools only), and their original order of 
selection (within stratum). From this sorted list, they were grouped into pairs within each sampling 
stratum; the last pair in the stratum may be a triplet if the number of schools in the stratum is odd. This 
operation resulted in a number of ordered preliminary variance strata of two or three units each. The first 
ordered 63 strata were then numbered sequentially from 1 to 63; the next ordered 63 strata were also 
numbered sequentially from 1 to 63, and so on until the list was exhausted, thus forming the desired 63 
variance strata. 

In strata with two units, a unit being a PSU in the case of NSR PSUs and a school in the case 
of SR PSUs, the base weight of the first unit was doubled to form the replicate weight, while the base 
weight of the second unit was multiplied by zero. In strata with three units, two variance strata were 
created: in the first variance stratum, the base weight of two of the three units was multiplied by 1.5 to 
form the replicate weight, and the base weight of the last unit was multiplied by zero; in the second 
variance stratum, the base weight of a different group of two units was multiplied by 1.5, and the base 
weight of the third unit was multiplied by zero. Multiplying the base weight in a unit by zero is equivalent 
to dropping one unit as required by the jackknife method. All adjustments to the lull sample weights were 
repeated for the replicate weights. For each full sample weight, there are 90 replicate weights with the 
same weight prefix. 

A child sampled in first grade through the freshening process was assigned to the same 
replicate as the originally sampled child to whom the child was linked. When the child sampled in first 
grade was assigned a full sample weight, he or she was assigned the replicate weights in the same manner. 



7. 2.6.2 Replicate Weights for Samples Involving Fall-First Grade 

For the two longitudinal weights involving fall-first grade (CT6SC0 and C16SP0), there 
are 40 replicate weights. The reason for the smaller number of replicates is that only a subsample of 
schools was included in the fall-first grade sample. The weights associated with the fall- first grade data do 
not account for the Durbin method of selecting PSUs, since it no longer applied. Rather, they reflect the 
fact that only one of the two sampled PSUs in the NSR strata was kept in the subsample. To account for 
this feature, pairs of similar NSR PSUs were collapsed into 19 variance strata. The SR PSUs account for 
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the remaining 2 1 variance strata. The 40 replicates will yield about 40 degrees of freedom for calculating 
confidence intervals for many survey estimates. 

7.3 Variance Estimation 

7.3.1 Jackknife Method 

The final full sample and the adjusted replicate weights can be used to compute estimates of 
variance for survey estimates using WesVar, AM, SUDAAN, or other software that handles replicate 
weights. The estimate of variance is the sum of the squared deviations of the replicate estimates from the 
full-sample estimate: 

g=l 

where 

0 is the population statistic of interest, 

9 is the estimate of 6 based on the full sample, 

G is the number of replicates, and 

9 (g) is the g th replicate estimate of 0 based on the observations included in the g th 
replicate. 

7.3.2 Taylor Series Method 

Variance stratum and variance unit (first-stage sample unit) identifiers were also created to 
be used in statistical software that computes variance estimates based on the Taylor series method (e.g., 
SUDAAN, Stata, SAS, SPSS Complex Samples Module, and AM). In this method, a linear 
approximation to a statistic is formed and then substituted into the formula for calculating the variance of 
a linear estimate appropriate for the sample design. 
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If Y = {Y 1 ,...,Y p ) denotes a p-dimensional vector of population parameters, Y = {y i ,...,Y ) is 
the corresponding vector of estimators based on a sample s of size n(s), 0 - g{Y) is the population 
parameter of interest, and 0 - gfhjis an estimator of 0 , then 



0-e = fpYl(y Y 

Sf ay, ' ' 



and 









j = i i = i 



The Taylor series method relies on a simplified procedure for estimating the variance for a 
linear statistic even with a complex sample design and is valid in large samples in which the first stage 
units are sampled with replacement. For the ECLS-K, this simplified method does not capture the 
variance related to the Durbin sampling method, the effects of the adjustments of the weights for 
nonresponse, or the sample-based raking procedures. These effects are not captured in the Taylor series 
variance estimates mainly because each adjustment corresponds to a different estimator that the variance 
estimation software does not support. In some cases these adjustments may have only a minor effect on 
the variance estimates, but in other cases the effects could be more substantial. 



For software that uses the Taylor series method, the variance strata and PSUs must be 
defined. For the fifth-grade ECLS-K, the Taylor variance strata were assigned by sequentially numbering 
the sampling strata and collapsing any stratum with one PSU with an adjacent stratum. In theory, any 
variance stratum with fewer than two responding units would be combined with an adjacent stratum, but 
this did not happen in the ECLS-K. The variance units were assigned by sequentially numbering the first- 
stage sampling units within sampling strata. For example, for C6CW0, Taylor variance strata were 
numbered sequentially from 1 to 90. Within each Taylor stratum, Taylor units were numbered 
sequentially from 1 to the total number of units in the stratum. This procedure was done separately for 
each cross-sectional and longitudinal weight. 
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APPENDIX A: MOVER ADJUSTMENT CELLS 



Variables used to create mover adjustment cells: 

FRESHEE = 0, sampled in kindergarten 

1 , sampled in first grade 

LMSTATUS = 1 , home language is not English 

2, home language is English 

These variables are in the weighting working files. If they are included in the final data file, they do not 
have the same variable names. 

Table A-l. Mover adjustment cells for cross-sectional weights C6CW0, C6PW0 and C6CPTR0: 
School year 2003-04 



Cell 


FRESHEE 


LMSTATUS 


Adjustment factor 


1 


0 


1 


2.18502 


2 


0 


2 


6.39362 


3 


1 


all 


2.46993 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



Table A-2. Mover adjustment cells for cross-sectional weight C6CPTM0: 
School year 2003-04 



Cell 


LMSTATUS 


Adjustment factor 


1 


1 


1.51759 


2 


2 


3.26127 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood 
Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 



Table A-3. Mover adjustment cells for cross-sectional weight C6CPTS0: 
School year 2003-04 



Cell 


LMSTATUS 


Adjustment factor 


1 


1 


1.48393 


2 


2 


3.38600 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood 
Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 
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Table A-4. Mover adjustment cells for longitudinal weights not involving fall-first grade: School 
year 2003-04 



Cell 


FRESHEE 


LMSTATUS 


Adjustment factor 


1 


0 


1 


2.11188 


2 


0 


2 


5.87163 


3 


1 


all 


2.35774 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten 
Class of 1998-99 (ECLS-K), spring 2004. 



Table A-5. Mover adjustment cells for longitudinal weights involving 
fall-first grade: School year 2003-04 



Cell 


LMSTATUS 


Adjustment factor 


1 


1 


2.04280 


2 


2 


4.42024 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood 
Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 
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Appendix B: UNKNOWN ELIGIBILITY AND NONRESPONSE ADJUSTMENT CELLS 



Variables used to create u nk nown eligibility and nonresponse adjustment cells: 

HHTYPE = 1 , in family of 2 parents plus siblings 

2, in family of 2 parents and no siblings 

3, in family of 1 parent plus siblings 

4, in family of 1 parent and no siblings 

5, in other type of family 

9, in u nk nown type of family 

HOMESCH = 0, not homeschooled 

1, in home school 

LFMOVST = 0, not a longitudinal (full sample) mover 

1 , longitudinal (full sample) mover 

LMSTATUS = 1 , home language is not English 

2, home language is English 

LSMOVST = 0, not a longitudinal (subsample) mover 

1 , longitudinal (subsample) mover 

R6MOVST = 0, not a spring-fifth grade mover 

1 , spring-fifth grade mover 

RC12P12 = 1, not assessed due to a disability or language minority (LM) or respondent to child 

assessment and parent interview in fall-kindergarten and spring-kindergarten (i.e., 
C 1 C W0>0 and C2CWOO and C 1 PW0>0 and C2PWOO) 

2, not a respondent to child assessment or parent interview in fall-kindergarten or 
spring-kindergarten (i.e., C1CW0=0 or C2CW0=0 or C1PW0=0 or C2PW0=0) 

RC1245P1245 = 1, not assessed due to a disability or language minority (LM) or respondent to child 
assessment and parent interview in fall-kindergarten and spring-kindergarten and 
spring-first grade and spring-third grade (i.e., C1CW0>0 and C2CWOO and 
C4CWOO and C5CW0X) and C1PW0X) and C2PW0X) and C4PW0X) and 
C5PWOO) 

2, not a respondent to child assessment or parent interview in fall-kindergarten or 
spring-kindergarten or spring-first grade or spring-third grade(i.e., C1CW0=0 or 
C2CW0=0 or C4CW0=0 or C5CW0=0 or C1PW0=0 or C2PW0=0 or C4PW0=0 or 
C5PW0=0) 

RC125P125 = 1, not assessed due to a disability or language minority (LM) or respondent to child 

assessment and parent interview in fall-kindergarten and spring-kindergarten and 
spring-third grade (i.e., C1CWOO and C2CWOO and C5CW0X) and C1PW0>0 and 
C2PWOO and C5PW0>0) 
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2, not a respondent to child assessment or parent interview in fall-kindergarten or 
spring-kindergarten or spring-third grade (i.e., C1CW0=0 or C2CW0=0 or C5CW0=0 
or C1PW0=0 or C2PW0=0 or C5PW0=0) 



RC5 = 1, not assessed due to a disability or LM or respondent to spring-third grade child 

assessment (i.e., C5CW0>0) 

2, not a respondent to spring-third grade child assessment (i.e., C5CW0=0) 

RP4 = 1, a respondent to parent interview in spring-first grade (i.e., C4PW0>0) 

2, not a respondent to parent interview in spring-first grade (i.e., C4PW0=0) 

RP5 = 1, a respondent to parent interview in spring-third grade (i.e., C5PWOO) 

2, not a respondent to parent interview in spring-third grade (i.e., C5PW0=0) 



STYPE = 1, Catholic 

2, Other religious private 

3, Nonreligious private 

4, public 

9, unknown 



These variables are in the weighting working files. If they are included in the final data file, they do not 
have the same variable names. 
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Table B-l. Unknown eligibility and nonresponse adjustment cells for C6CW0: School year 2003-04 



Cell 


R6MOVST 


RC125P125 


RP4 


LMSTATUS 


Adjustment factor 
Unknown 

eligibility Nonresponse 


1 


0 


1 


all 


All 


1.00000 


1.00520 


2 


0 


2 


1 


All 


1.00780 


1.04758 


3 


0 


2 


2 


1 


1.45563 


1.01915 


4 


0 


2 


2 


2 


2.02103 


1.13393 


5 


1 


1 


all 


All 


1.04393 


1.19431 


6 


1 


2 


all 


All 


1.12945 


1.80780 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



Table B-2. Unknown eligibility and nonresponse adjustment cells for C6PW0: School year 2003-04 



Adjustment factor 



Cell 


R6MOVST 


HHTYPE 


RP5 


LMSTATUS 


RC5 


Unknown 

eligibility 


Nonresponse 


1 


0 


1,2, 4,5 


1 


1 


all 


1.00000 


1.06209 


2 


0 


1,2, 4,5 


1 


2 


all 


1.00000 


1.03967 


3 


0 


1,2, 4,5 


2 


1 


all 


1.26456 


1.38246 


4 


0 


1,2, 4,5 


2 


2 


all 


1.40014 


1.57344 


5 


0 


3,9 


1 


all 


all 


1.00000 


1.08564 


6 


0 


3,9 


2 


1 


all 


1.29265 


1.37617 


7 


0 


3,9 


2 


2 


all 


1.79527 


1.85378 


8 


1 


1,2, 4,5 


1 


all 


1 


1.03383 


1.07452 


9 


1 


1,2, 4,5 


1 


all 


2 


1.05715 


1.14946 


10 


1 


1,2, 4,5 


2 


all 


all 


1.28633 


1.60367 


11 


1 


3,9 


1 


all 


1 


1.08367 


1.11359 


12 


1 


3,9 


1 


all 


2 


1.17165 


1.02198 


13 


1 


3,9 


2 


all 


all 


1.16237 


1.81606 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



B-3 








Table B-3. Unknown eligibility and nonresponse adjustment cells for C6CPTR0: School year 2003-04 



Cell HOMESCH R6MOVST 


HHTYPE 


RP5 


LMSTATUS 


STYPE 


Adjustment factor 
Unknown 

eligibility Nonresponse 


1 


1 


all 


all 


all 


all 


all 


1.00000 


1.58484 


2 


0 


0 


1,2, 4, 5 


1 


1 


1,2,3 


1.00000 


1.07445 


3 


0 


0 


1,2, 4, 5 


1 


1 


4,9 


1.00000 


1.12005 


4 


0 


0 


1,2, 4, 5 


1 


2 


1,2,3 


1.00000 


1.04118 


5 


0 


0 


1,2, 4, 5 


1 


2 


4,9 


1.00000 


1.06871 


6 


0 


0 


1,2, 4, 5 


2 


1 


all 


1.26456 


1.45233 


7 


0 


0 


1,2, 4, 5 


2 


2 


all 


1.40014 


1.64009 


8 


0 


0 


3,9 


1 


all 


all 


1.00000 


1.10832 


9 


0 


0 


3,9 


2 


1 


all 


1.29265 


1.48494 


10 


0 


0 


3,9 


2 


2 


all 


1.79527 


1.89980 


11 


0 


1 


all 


all 


all 


1,2,3 


1.00000 


1.21747 


12 


0 


1 


all 


all 


1 


4,9 


1.11289 


1.64403 


13 


0 


1 


all 


all 


2 


4,9 


1.08163 


1.84413 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 
(ECLS-K), spring 2004. 



Table B-4. Unknown eligibility and nonresponse adjustment cells for C6CPTM0: School year 2003-04 



Adjustment factor 



Cell HOMESCH R6MOVST 


HHTYPE 


STYPE 


RP5 


LMSTATUS 


Unknown 

eligibility 


Nonresponse 


1 


1 


All 


all 


all 


all 


all 


1.00000 


2.07187 


2 


0 


0 


1,5 


1,2,3 


all 


1 


1.00000 


1.08125 


3 


0 


0 


1,5 


1,2,3 


all 


2 


1.00000 


1.05655 


4 


0 


0 


1,5 


4,9 


1 


1 


1.00000 


1.11565 


5 


0 


0 


1,5 


4,9 


1 


2 


1.00000 


1.07240 


6 


0 


0 


1,5 


4,9 


2 


1 


1.00000 


1.43444 


7 


0 


0 


1,5 


4,9 


2 


2 


1.00000 


1.62500 


8 


0 


0 


2, 3, 4, 9 


1,2,3 


all 


1 


1.00000 


1.35998 


9 


0 


0 


2, 3, 4, 9 


1,2,3 


all 


2 


1.00000 


1.16894 


10 


0 


0 


2, 3, 4, 9 


4,9 


1 


all 


1.00000 


1.10150 


11 


0 


0 


2, 3, 4, 9 


4,9 


2 


1 


1.00000 


1.39477 


12 


0 


0 


2, 3, 4, 9 


4,9 


2 


2 


1.00000 


1.55784 


13 


0 


1 


all 


1,2,3 


all 


all 


1.00000 


1.15601 


14 


0 


1 


all 


4,9 


1 


1 


1.07308 


1.51809 


15 


0 


1 


all 


4,9 


1 


2 


1.04336 


1.87984 


16 


0 


1 


all 


4,9 


2 


all 


1.20402 


2.13094 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 
(ECLS-K), spring 2004. 
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Table B-5. Unknown eligibility and nonresponse adjustment cells for C6CPTS0: School year 2003-04 



Cell HOMESCH R6MOVST 


RP5 


STYPE 


LMSTATUS 


HHTYPE 


Adjustment factor 
Unknown 

eligibility Nonresponse 


1 


l 


All 


all 


all 


all 


all 


1.00000 


1.10612 


2 


0 


0 


1 


1,2,3 


1 


all 


1.00000 


1.06891 


3 


0 


0 


1 


1,2,3 


2 


all 


1.00000 


1.04568 


4 


0 


0 


1 


4,9 


1 


1,2,3 


1.00000 


1.11746 


5 


0 


0 


1 


4,9 


1 


4,5,9 


1.00000 


1.20287 


6 


0 


0 


1 


4,9 


2 


1,2,4 


1.00000 


1.06330 


7 


0 


0 


1 


4,9 


2 


3,5,9 


1.00000 


1.10587 


8 


0 


0 


2 


all 


1 


all 


1.00000 


1.47316 


9 


0 


0 


2 


all 


2 


all 


1.00000 


1.83441 


10 


0 


1 


all 


1,2,3 


all 


all 


1.00000 


1.28755 


11 


0 


1 


all 


4,9 


1 


all 


1.11629 


1.69071 


12 


0 


1 


all 


4,9 


2 


all 


1.09946 


1.81234 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 



(ECLS-K), spring 2004. 
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Table B-6. Unknown eligibility and nonresponse adjustment cells for C56CW0: 
School year 2003-04 



Adjustment factor 



Cell 


LFMOVST 


RC125P125 


LMSTATUS 


Unknown 

eligibility 


Nonresponse 


1 


0 


1 


all 


1.00000 


1.00499 


2 


0 


2 


all 


1.20681 


1.09388 


3 


1 


1 


all 


1.04013 


1.17679 


4 


1 


2 


1 


1.16114 


1.81191 


5 


1 


2 


2 


1.09921 


2.14838 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal 
Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 



Table B-7. Unknown eligibility and nonresponse adjustment cells for C56PW0: School year 2003-04 



Cell 


LFMOVST 


RC125P125 


STYPE 


LMSTATUS 


HHTYPE 


Adjustment factor 
Unknown 

eligibility Nonresponse 


1 


0 


1 


1,2,3 


all 


all 


1.00000 


1.02039 


2 


0 


1 


4,9 


all 


all 


1.00000 


1.04605 


3 


0 


2 


all 


1 


all 


1.14175 


2.19706 


4 


0 


2 


all 


2 


all 


1.22822 


1.86073 


5 


1 


1 


all 


all 


all 


1.04013 


1.07404 


6 


1 


2 


all 


all 


1,4,5 


1.10370 


1.66966 


7 


1 


2 


all 


all 


2,3,9 


1.13694 


2.13831 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1 998— 
99 (ECLS-K), spring 2004. 



Table B-8. Unknown eligibility and nonresponse adjustment cells for C456CW0: School year 2003-04 















Adjustment factor 


Cell 


LFMOVST 


RC1245P1245 


STYPE 


RC12P12 


LMSTATUS 


Unknown 

eligibility 


Nonresponse 


1 


0 


1 


all 


all 


all 


1.00000 


1.00498 


2 


0 


2 


all 


1 


all 


1.06836 


1.27287 


3 


0 


2 


all 


2 


all 


1.25666 


1.09852 


4 


1 


all 


1,2,3 


all 


all 


1.00000 


1.18872 


5 


1 


all 


4,9 


1 


all 


1.07156 


1.51868 


6 


1 


all 


4,9 


2 


1 


1.13642 


1.63119 


7 


1 


all 


4,9 


2 


2 


1.08907 


1.97756 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 
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Table B-9. Unknown eligibility and nonresponse adjustment cells for C456PW0: School year 2003-04 



Cell 


LFMOVST 


LMSTATUS 


HHTYPE 


STYPE 


RC12P12 


Adjustment factor 
Unknown 

eligibility Nonresponse 


1 


0 


1 


1,5 


all 


all 


1.03557 


1.29913 


2 


0 


1 


2, 3, 4, 9 


all 


all 


1.06620 


1.51069 


3 


0 


2 


1,2 


1,2,3 


all 


1.04308 


1.09147 


4 


0 


2 


1,2 


4,9 


all 


1.02968 


1.17355 


5 


0 


2 


3,4, 5,9 


all 


all 


1.12449 


1.36726 


6 


1 


1 


1 


all 


all 


1.08017 


1.41610 


7 


1 


1 


2,3,4, 5,9 


all 


all 


1.13332 


1.77362 


8 


1 


2 


1,2, 4,5 


all 


1 


1.04744 


1.22519 


9 


1 


2 


1,2, 4,5 


all 


2 


1.09761 


1.73074 


10 


1 


2 


3,9 


all 


1 


1.10180 


1.53178 


11 


1 


2 


3,9 


all 


2 


1.06019 


2.05338 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 
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Table B-10. Unknown eligibility and nonresponse adjustment cells for C2 6FC0: School year 2003-04 



Cell 


LFMOVST 


RC1245P1245 


STYPE 


HHTYPE 


RC12P12 


Adjustment factor 
Unknown 

eligibility Nonresponse 


1 


0 


1 


all 


all 


all 


1.00000 


1.00498 


2 


0 


2 


1,2,3 


1,2,5 


1 


1.10270 


1.27759 


3 


0 


2 


1,2,3 


1,2,5 


2 


1.23869 


1.08868 


4 


0 


2 


1,2,3 


3,4,9 


all 


1.48098 


1.37939 


5 


0 


2 


4,5,9 


1,2,5 


1 


1.06672 


1.33018 


6 


0 


2 


4,5,9 


1,2,5 


2 


1.17624 


1.14951 


7 


0 


2 


4,5,9 


3,4,9 


all 


1.30405 


1.17559 


8 


1 


all 


all 


1,2,3 


all 


1.07001 


1.53865 


9 


1 


all 


all 


4,5,9 


all 


1.09007 


1.95635 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 



1998-99 (ECLS-K), spring 2004. 



Table B-ll. Unknown eligibility and nonresponse adjustment cells for C2 6FP0: School year 
2003-04 



Cell 


LFMOVST 


LMSTATUS 


HHTYPE 


STYPE 


Adjustment factor 
Unknown 

eligibility Nonresponse 


1 


0 


1 


all 


all 


1.04424 


1.37476 


2 


0 


2 


all 


1,2,3 


1.06020 


1.11509 


3 


0 


2 


1-2 


4,9 


1.03026 


1.19437 


4 


0 


2 


3-9 


4,9 


1.11064 


1.40976 


5 


1 


1 


1 


all 


1.07866 


1.43354 


6 


1 


1 


2-9 


all 


1.13166 


1.86774 


7 


1 


2 


1,2, 4,5 


all 


1.05586 


1.33576 


8 


1 


2 


3,9 


all 


1.09101 


1.74837 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, 
Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 
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Table B-12. U nk nown eligibility and nonresponse adjustment cells for Cl 6FC0: School year 
2003-04 



Cell 


LFMOVST 


STYPE 


RC1245P1245 


LMSTATUS 


Adjustment factor 
Unknown 

eligibility Nonresponse 


1 


0 


1,2,3 


all 


all 


1.06076 


1.19946 


2 


0 


4,9 


1 


all 


1.00000 


1.00476 


3 


0 


4,9 


2 


1 


1.12199 


1.48187 


4 


0 


4,9 


2 


2 


1.20797 


1.86898 


5 


1 


1,2,3 


all 


all 


1.00000 


1.29333 


6 


1 


4,9 


all 


1 


1.10195 


1.65526 


7 


1 


4,9 


all 


2 


1.07065 


1.77467 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, 
Kindergarten Class of 1998-99 third (ECLS-K), spring 2004. 



Table B-13. Unknown eligibility and nonresponse adjustment cells for Cl 6FP0: School year 
2003-04 



Adjustment factor 



Cell 


LFMOVST 


LMSTATUS 


HHTYPE 


STYPE 


Unknown 

eligibility 


Nonresponse 


1 


0 


1 


all 


all 


1.04424 


1.51744 


2 


0 


2 


1-2 


all 


1.03228 


1.29147 


3 


0 


2 


3-9 


all 


1.11400 


1.56542 


4 


1 


1 


1 


all 


1.07866 


1.60947 


5 


1 


1 


2-9 


all 


1.13166 


2.08877 


6 


1 


2 


all 


1,2,3 


1.00000 


1.33747 


7 


1 


2 


1,2,5 


4,9 


1.06203 


1.42833 


8 


1 


2 


3,4,9 


4,9 


1.08873 


1.94624 


SOURCE 


: U.S. Department 


of Education, National 


Center for Education 


Statistics, 


Early Childhood Longitudinal Study, 



Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 
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Table B-14. Unknown eligibility and nonresponse adjustment cells for 
C1_6SC0: School year 2003-04 











Adjustment factor 


Cell 


LSMOVST 


STYPE 


HHTYPE 


Unknown 

eligibility 


Nonresponse 


1 


0 


1,2,3 


all 


1.08002 


1.10377 


2 


0 


4,9 


1,3,5 


1.04662 


1.14636 


3 


0 


4,9 


2, 4,9 


1.20938 


1.29705 


4 


1 


1,2,3 


all 


1.00000 


1.30250 


5 


1 


4,9 


all 


1.06725 


1.86940 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood 
Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 



Table B-15. U nk nown eligibility and nonresponse adjustment cells for Cl 6SP0: School year 
2003-04 



Cell 


LSMOVST 


LMSTATUS 


STYPE 


HHTYPE 


Adjustment factor 
Unknown 

eligibility Nonresponse 


1 


0 


1 


all 


all 


1.04988 


1.58359 


2 


0 


2 


1,2,3 


all 


1.07635 


1.25392 


3 


0 


2 


4,9 


1,2 


1.04169 


1.30604 


4 


0 


2 


4,9 


3-9 


1.15726 


1.64778 


5 


1 


all 


1,2,3 


all 


1.00000 


1.29231 


6 


1 


all 


4,9 


1,2 


1.06374 


1.56525 


7 


1 


all 


4,9 


3-9 


1.07412 


2.20842 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal 
Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 
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APPENDIX C: RAKING DIMENSIONS 



The raking dimensions are in the weighting working files. They are not included in any delivery files. 
Table C-l. Raking dimension 1 — sex by age: School year 2003-04 



DIM1 


1 NEWGENDER NEWAGE 


Description 


11 


1 


l 


Male; age 7 or 8 


12 


1 


2 


Male; age 9 or missing 


13 


1 


3 


Male; age 10 or 1 1 


21 


2 


1 


Female; age 7 or 8 


22 


2 


2,3 


Female; age 9, 10, 11 or missing 


‘For Cl 


6SC0 and Cl 


6SP0, cell 13 was merged with cell 12. 


SOURCE: 


: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal 


Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 





Table C-2. 


Raking dimension 2- 


—region by locale: School year 2003-04 


DIM2 


NEWREGION 


NEWLOCALE Description 


11 


1 


1 


Northeast; central city 


12 


1 


2 


Northeast; suburb/large town 


13 


1 


3 


Northeast; small town/rural 


21 


2 


1 


Midwest; central city 


22 


2 


2 


Midwest; suburb/large town 


23 


2 


3 


Midwest; small town/rural 


31 


3 


1 


South; central city 


32 


3 


2 


South; suburb/large town 


33 


3 


3 


South; small town/rural 


41 


4 


1 


West; central city 


42 


4 


2 


West; suburb/large town 


43 


4 


3 


West; small town/rural 


SOURCE: U.S. 


Department of Education, National Center for Education Statistics, Early Childhood Longitudinal 



Study, Kindergarten Class of 1 998-99 (ECLS-K), spring 2004. 
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Table C-3. Raking dimension 3 — race/ethnicity by SES quintile: School year 2003-04 



DIM3 


NEWRACE 


NEWSESQ5 


Description 


11 


1 


1 


White; SESQ1 


12 


1 


2 


White; SES Q2 


13 


1 


3 


White; SES Q3 


14 


1 


4 


White; SES Q4 


15 


1 


5 


White; SES Q5 


21 


2 


1 


Black; SES Q1 


22 


2 


2 


Black; SES Q2 


23 


2 


3 


Black; SES Q3 


24 


2 


4,5 


Black; SES Q4 or Q5 


31 


3 


1 


Hispanic; SES Q1 


32 


3 


2 


Hispanic; SES Q2 


33 


3 


3 


Hispanic; SES Q3 


34 


3 


4 


Hispanic; SES Q4 


35 


3 


5 


Hispanic; SES Q5 


41 


4 


1,2 


API; SESQ1 or Q2 


42 


4 


3,4 


API; SES Q3 or Q4 


43 


4 


5 


API; SES Q5 


51 


5 


all 


American Indian/Alaskan 


61 


6 


all 


Other race 



Tor C1_6SC0 and C16SP0, cell 23 was merged with cell 22; cell 35 was merged with cell 34; cells 42 
and 43 were merged with cell 41 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal 
Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 



Table C-4. Raking dimension 4 — school affiliation: School 
year 2003-04 



DIM4 


NEWSTYPE 


Description 


10 


1 


Catholic 


20 


2 


Other religious private 


30 


3 


Nonreligious private 


40 


4 


Public 



SOURCE: U.S. Department of Education, National Center for Education 
Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 
(ECLS-K), spring 2004. 



C-2 





Table C-5. Raking dimension 5 — language minority status: 
School year 2003-04 



DIM5 


LMSTATUS 


Description 




10 


1 


Language minority 




20 


2 


Non-language minority 




SOURCE: U.S. Department of Education, National Center for Education Statistics, Early 
Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 




Table C-6. 


Raking dimension 6- 
year 2003-04 


—mover status in spring- first grade: 


School 


DIM6 1 


MOVERST 


Description 




10 


0 


Sampled in KG, spring-first grade 


nonmover 


20 


1 


Sampled in KG, spring first-grade 


mover 


30 


9 


Sampled in first grade 





'For longitudinal weights involving spring-kindergarten data, cell 30 was not applicable. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 
1998-99 (ECLS-K), spring 2004. 



C-3 



This page is intentionally left blank. 



C-4 




